OK. So you want to prepopulate the Pulldown using an argument from the URLLoader. No problem.
There's an example in Sideview Utils about this (see the 'linking' section), but I've been meaning to give it some friends and expand on it a bit more. For now you can see the three examples of how that page links to other pages and how the prepopulation works there in the target pages.
The main change you have to make is dont call the argument 'clicked'. I know you're trying to make it simpler but just call it whatever the name of your Pulldown is and you'll be back on the right track. You do not need ValueSetter at all. All you have to do is have a key like 'foo=12' in the URL, URLLoader will make that key value pair available downstream, and if there's a Pulldown or TextField whose name param is 'foo', it will prepopulate.
Some more notes:
there is no 'searchWhenChanged' param on ValueSetter.
you have to put Search module downstream from all of the Pulldowns/TextFields/Checkbox/SearchBar modules that contribute to it. In your XML you have placed the Search upstream from the Pulldown which isnt what you want. (There is some documentation in sideview utils mentioning this as a key difference between HiddenSearch and Search)
You can use $name$ as a shorthand in the 'searchFieldsToDisplay' param, but putting other context keys here is pretty advanced and I doubt it's really what you want -- what you had would look for a field whose name is the value of 'passdownClicked', and it's unlikely such a field name exists.
In searchFieldsToDisplay and staticFieldsToDisplay, if value and label are the same you can just specify the value.
Its not necessary to specify the $earliest$ and $latest$ in the Search module. I have removed this because the TimeRangePicker module will pick those up from URLLoader automatically.
The end result
<module name="URLLoader" layoutPanel="panel_row1_col1" autoRun="True">
<!-- this Search just feeds the Pulldown -->
<param name="search">index=test $dbs$ | stats count by Server</param>
<param name="postProcess">dedup $name$ | sort $name$</param>
<param name="search">index=test $dbs$ $Server$</param>
<param name="html">The passed down value is $Server$</param>
Again, the main change is that the parameter in the URL should be Server=something, not clicked=something. If for some reason the value has to be different, you could patch it with a ValueSetter to convert the 'clicked' key to a 'Server' key but that would be an extra step. It's way easier to just make the names match up.
I recommend taking another pass reading through the relevant examples in the Sideview Utils docs, and I will make a note to improve the examples here for clarity and completeness.