Getting Data In

How to parse and index fields from my unstructured data?

minkyuk
Explorer

Hi,

I'm trying to successfully parse out some fields from unstructured log file.
Below is a snippet:


Tue Jun 16 00:15:27 EDT 2015 
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 
root 2 0.0 0.0 0 0 ? S Jun07 0:00 [kthreadd] 
root 3 0.0 0.0 0 0 ? S Jun07 1:06 \_ [mi/0] 
root 4 0.0 0.0 0 0 ? S Jun07 0:15 \_ [ks/0] 
root 5 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/0] 
root 6 0.0 0.0 0 0 ? S Jun07 5:27 \_ [wa/0] 
root 7 0.0 0.0 0 0 ? S Jun07 1:39 \_ [mi/1] 
root 8 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/1] 
root 9 0.0 0.0 0 0 ? S Jun07 0:14 \_ [ks/1] 
root 10 0.0 0.0 0 0 ? S Jun07 0:01 \_ [wa1]
root 11 0.0 0.0 0 0 ? S Jun07 1:04 \_ [mi/2] 
root 12 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/2]
-----------------------------------------
Tue Jun 16 00:20:27 EDT 2015 
....

Using Splunk data parser, how could I patternize and successfully get a specific column or two?
(I am looking into ways to find smart patterns using regex, or just ------------------- as a pattern)

Thanks,
Jack

0 Karma
1 Solution

woodcock
Esteemed Legend

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

View solution in original post

0 Karma

woodcock
Esteemed Legend

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

0 Karma

prsak1
New Member

Hi ,

Can you please provide a demo for unstructured data.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

@prsak1 You're adding on to a question that is more than three years old and has an accepted answer. There's not likely to be many people seeing your comment. I suggest you post a new question describing the problem you are trying to resolve.

---
If this reply helps you, Karma would be appreciated.
0 Karma

minkyuk
Explorer

Thank you for detailed response; where could I edit props.conf?

0 Karma

woodcock
Esteemed Legend

The same place where you edited inputs.conf.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have a look at the multikv command.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...