How to parse and index fields from my unstructured data?

[minkyuk]

Hi,

I'm trying to successfully parse out some fields from unstructured log file.
Below is a snippet:

---

Tue Jun 16 00:15:27 EDT 2015 
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 
root 2 0.0 0.0 0 0 ? S Jun07 0:00 [kthreadd] 
root 3 0.0 0.0 0 0 ? S Jun07 1:06 \_ [mi/0] 
root 4 0.0 0.0 0 0 ? S Jun07 0:15 \_ [ks/0] 
root 5 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/0] 
root 6 0.0 0.0 0 0 ? S Jun07 5:27 \_ [wa/0] 
root 7 0.0 0.0 0 0 ? S Jun07 1:39 \_ [mi/1] 
root 8 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/1] 
root 9 0.0 0.0 0 0 ? S Jun07 0:14 \_ [ks/1] 
root 10 0.0 0.0 0 0 ? S Jun07 0:01 \_ [wa1]
root 11 0.0 0.0 0 0 ? S Jun07 1:04 \_ [mi/2] 
root 12 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/2]
-----------------------------------------
Tue Jun 16 00:20:27 EDT 2015 
....

Using Splunk data parser, how could I patternize and successfully get a specific column or two?
(I am looking into ways to find smart patterns using regex, or just ------------------- as a pattern)

Thanks,
Jack

[woodcock · Accepted solution]

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

[prsak1]

Hi ,

Can you please provide a demo for unstructured data.

[richgalloway]

@prsak1 You're adding on to a question that is more than three years old and has an accepted answer. There's not likely to be many people seeing your comment. I suggest you post a new question describing the problem you are trying to resolve.

[minkyuk]

Thank you for detailed response; where could I edit props.conf?

[woodcock]

The same place where you edited inputs.conf.

[richgalloway]

Have a look at the multikv command.