Getting Data In

How to parse and index fields from my unstructured data?

minkyuk
Explorer

Hi,

I'm trying to successfully parse out some fields from unstructured log file.
Below is a snippet:


Tue Jun 16 00:15:27 EDT 2015 
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 
root 2 0.0 0.0 0 0 ? S Jun07 0:00 [kthreadd] 
root 3 0.0 0.0 0 0 ? S Jun07 1:06 \_ [mi/0] 
root 4 0.0 0.0 0 0 ? S Jun07 0:15 \_ [ks/0] 
root 5 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/0] 
root 6 0.0 0.0 0 0 ? S Jun07 5:27 \_ [wa/0] 
root 7 0.0 0.0 0 0 ? S Jun07 1:39 \_ [mi/1] 
root 8 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/1] 
root 9 0.0 0.0 0 0 ? S Jun07 0:14 \_ [ks/1] 
root 10 0.0 0.0 0 0 ? S Jun07 0:01 \_ [wa1]
root 11 0.0 0.0 0 0 ? S Jun07 1:04 \_ [mi/2] 
root 12 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/2]
-----------------------------------------
Tue Jun 16 00:20:27 EDT 2015 
....

Using Splunk data parser, how could I patternize and successfully get a specific column or two?
(I am looking into ways to find smart patterns using regex, or just ------------------- as a pattern)

Thanks,
Jack

0 Karma
1 Solution

woodcock
Esteemed Legend

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

View solution in original post

0 Karma

woodcock
Esteemed Legend

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

View solution in original post

0 Karma

prsak1
New Member

Hi ,

Can you please provide a demo for unstructured data.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

@prsak1 You're adding on to a question that is more than three years old and has an accepted answer. There's not likely to be many people seeing your comment. I suggest you post a new question describing the problem you are trying to resolve.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

minkyuk
Explorer

Thank you for detailed response; where could I edit props.conf?

0 Karma

woodcock
Esteemed Legend

The same place where you edited inputs.conf.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have a look at the multikv command.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!