Using inputlookup with a field I have to REX.

[mrgibbon]

Hey Guys, I have a list of hostnames in a CSV file, I need to run a search in Splunk which references this list. Easy, I can do that.
But what I cant get my head around is how I can compare that list against a field which I am manually running a rex command to create.

Basically the "hostname" field doesnt exist in the search results. I have the list of hostnames in the CSV file. How do I search on those hostnames, without using transforms or anything else tricky on the backend?

This is my latest try, it doesnt produce any results.
source="*xxx.tmp" [|inputlookup xxx.csv | fields hostname ] | rex "(?i),Info,(?P[^,]+)" | table hostname

Thanks in advance.

[mrgibbon]

Thanks Drainy, Thats some good advice there, thanks.
I will try and put it into a transform I think, seems to be the best way.
Will that transform be applied at searching time, or only on new records that come in after I apply it? Cheers!

Here is an example of the data:
2013-02-17 14:32:48,Info,CC-CXXXXXX,Category: 0,The client has successfully downloaded and applied a license file from the server.

Where CC-CXXXXXX is the hostname.

[Drainy]

We could really do with some example data to really help but whatever you do you aren't going to be able to compare them without performing an extraction on the host name in the event.

Putting this in transforms/props isn't at all tricky but will leave you with a host field that is extracted each time you run a search (without doing rex).

So, there is something in the event that links it to a host? To match these you need to have a common field. Forget inputlookup for now, I don't think thats the right command for what you want to do (it just throws the CSV onto the end of your results).

Lets say you had an event like this in mydata.log;

192.168.2.1 01/01/12 14:30 Error: I'm an event! With no meaning! Danger! 

Now you want to match the IP to the host in your CSV. Assuming you have the IP being extracted somewhere by rex or a rex in transforms/props, lets build a CSV...

ip,hostname
192.168.2.1,DC

and we'll call this hostnames.csv

Then in the search I would do a search for;

source=mydata.log | lookup hostnames.csv ip OUTPUT hostname

This would look for hostnames.csv either in the lookup directory of the current app you're in or for a global csv of the same name, it would pass in the ip and for any matches it would throw out the hostname.

This can be taken a step further by writing this straight into transforms/props so every time you run a search it does this, thats where the power of Splunk comes into play.

You could also do this via DNS if you really wanted but it depends on the use-case.
http://blogs.splunk.com/2009/12/15/reverse-dns-lookups-for-host-entries/

Post more details if this is not what you were talking about.