topic Re: NOOB - run a query in batch with varying input in Getting Data In

NOOB - run a query in batch with varying input

ranshe — Thu, 15 Dec 2011 00:08:13 GMT

Hi,
A total Splunk NOOB, so please go easy.

Anyway, every night I receive a bunch of numbers and I want Splunk to dig through my logs and tell me if these numbers are not there, or if they are tell me the info associated with these numbers.
How do I go about achieving that?

Thanks,

Re: NOOB - run a query in batch with varying input

lguinn2 — Thu, 15 Dec 2011 00:17:03 GMT

It would be helpful if you could give us some examples of your data (fake the ip addresses, etc. if needed).

You need to run a search, and then save it and schedule it to run every night. Schedule it to run at a time after you receive the numbers. Also, set it up to alert you via email "always" and it will send you a report every night that shows if anything matched.

Re: NOOB - run a query in batch with varying input

ranshe — Thu, 15 Dec 2011 05:31:39 GMT

Thanks lguinn.
My question is really about the "You need to run a search, and then save it and schedule it to run every night" part - how do I do that?
Also, is there a way for me to dump a file with my numbers and have Splunk read it and act on it?

Re: NOOB - run a query in batch with varying input

Ayn — Thu, 15 Dec 2011 06:53:34 GMT

This can certainly be done. The learning curve of implementing it might become a bit steep if you're totally new to Splunk though. 🙂

You should be able to achieve what you want using a combination of

A subsearch works a lot like backticks in UNIX if you're familiar with them. It runs first of all and passes on its results as arguments to the main search. A lookup can be either static, a CSV file that Splunk reads from, or dynamic, in which case Splunk calls a script and uses its results.

The idea in your case would be to have a lookup CSV file containing all the numbers you want to monitor, and then pass those numbers to the main search which checks if the numbers are there or not. Then, create an alert (http://docs.splunk.com/Documentation/Splunk/4.2.4/User/SchedulingSavedSearches) that triggers if you're getting results back.

My best advice from here would be to go read up on the mechanisms involved before diving into this. Your final search command might look something like this:

sourcetype="yoursourcetypewiththenumbers" [| inputlookup numbers.csv | fields number]

Here, I set the sourcetype to just a dummy sourcetype - I don't know how you filter on the exact logs that you want to check these numbers for. I also assumed that the numbers exist in a field called "numbers" in both the CSV file and in the logs - change this to whatever fieldname you're using. Hope this helps with getting you started.

Re: NOOB - run a query in batch with varying input

ranshe — Thu, 15 Dec 2011 19:27:05 GMT

Wow, lots of great info - highly appreciated!
Now the "homework" is on me - I'll play around and see how to use all this info.

Thanks.