topic Re: Getting nth line of a file in Splunk Search

Getting nth line of a file

bleinfelder — Wed, 30 Apr 2014 15:45:40 GMT

Hi there,

I'm trying to monitor the output of a batch job with splunk. The problem is that logging is poorly implemented and I need some sophisticated splunk (?) to meet my requirements.

The job writes errors to a log file. Together with the error message it states the line of the input file where the error occured. The input file can be splunked as well.

To be able to handle the error the guys in business department need the error message as well as the corresponding line in the input file.

I can extract the message with the line number from the job's log file.

How can I extract the correct line from the input file?

So far I tried to parse each line of the input file as a different event and getting the nth one using streamstats. Unfortunately the result stream doesnt maintain the correct order of the lines.

Any ideas about how I could extract the nth line of the file?

Regards,

Bernd

Re: Getting nth line of a file

somesoni2 — Wed, 30 Apr 2014 15:49:49 GMT

sample logs please.

Re: Getting nth line of a file

richgalloway — Wed, 30 Apr 2014 18:52:22 GMT

When you say "the input file can be splunked as well" does that mean you have control over how it is ingested? If so, include the line number as a field with each line of input. Then you can easily associate the log entry with the input line.

Re: Getting nth line of a file

martin_mueller — Wed, 30 Apr 2014 22:45:12 GMT

You can select a specific line from an event like this:

sourcetype=input_file
| eval lines = _raw | makemv tokenizer="(.+)" lines
| eval error_line = mvindex(lines, [stats count | eval line_number = 4 | return $line_number])

I'm assuming your input file is indexed as one large event. The tokenizer turns the copy of _raw into a multivalue field, with each entry corresponding to one line. mvindex then accesses the line you need, as calculated by the subsearch that returns the line number from your error message.

Re: Getting nth line of a file

bleinfelder — Wed, 07 May 2014 15:01:56 GMT

thank you, this works perfect!