Getting Data In

How to parse and index fields from my unstructured data?

minkyuk
Explorer

Hi,

I'm trying to successfully parse out some fields from unstructured log file.
Below is a snippet:


Tue Jun 16 00:15:27 EDT 2015 
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 
root 2 0.0 0.0 0 0 ? S Jun07 0:00 [kthreadd] 
root 3 0.0 0.0 0 0 ? S Jun07 1:06 \_ [mi/0] 
root 4 0.0 0.0 0 0 ? S Jun07 0:15 \_ [ks/0] 
root 5 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/0] 
root 6 0.0 0.0 0 0 ? S Jun07 5:27 \_ [wa/0] 
root 7 0.0 0.0 0 0 ? S Jun07 1:39 \_ [mi/1] 
root 8 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/1] 
root 9 0.0 0.0 0 0 ? S Jun07 0:14 \_ [ks/1] 
root 10 0.0 0.0 0 0 ? S Jun07 0:01 \_ [wa1]
root 11 0.0 0.0 0 0 ? S Jun07 1:04 \_ [mi/2] 
root 12 0.0 0.0 0 0 ? S Jun07 0:00 \_ [mi/2]
-----------------------------------------
Tue Jun 16 00:20:27 EDT 2015 
....

Using Splunk data parser, how could I patternize and successfully get a specific column or two?
(I am looking into ways to find smart patterns using regex, or just ------------------- as a pattern)

Thanks,
Jack

0 Karma
1 Solution

woodcock
Esteemed Legend

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

View solution in original post

0 Karma

woodcock
Esteemed Legend

You need to tell Splunk that this file has multi-line events like this in your props.conf file:

TIME_FORMAT=%a %b %d %H:%M:%S %Z %Y`
SHOULD_LINEMERGE= true
BREAK_ONLY_BEFORE_DATE = true

Then you need to tell Splunk that each event is of type 'multikv`.

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Multikvconf

There is also a multikv command:

http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/multikv

0 Karma

prsak1
New Member

Hi ,

Can you please provide a demo for unstructured data.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

@prsak1 You're adding on to a question that is more than three years old and has an accepted answer. There's not likely to be many people seeing your comment. I suggest you post a new question describing the problem you are trying to resolve.

---
If this reply helps you, Karma would be appreciated.
0 Karma

minkyuk
Explorer

Thank you for detailed response; where could I edit props.conf?

0 Karma

woodcock
Esteemed Legend

The same place where you edited inputs.conf.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have a look at the multikv command.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...