Getting Data In

How do you parse multiline key value events?

vishaltaneja070
Motivator

How do you parse the below events?

The events looks like :

2018-12-04 01:51:08.330, LogDate="2018-12-04 01:51:08.33", SessionId="abc", MachineName="xyz", LoggerName="def", LogLevel="DEBUG", MessageId="DumpCacheNames", Message="def", ApplicationId="fgd", EndpointStack="abc", LogInsertDate="date"

I need to show events as:

 "2018-12-04 01:51:08.330"
 SessionId=abc 
 MachineName=xyz 
 LoggerName=def
 LogLevel=DEBUG
 MessageId=DumpCacheNames
 Message=def
 ApplicationId=fgd
 EndpointStack=abc
 LogInsertDate=date

Thanks !

Tags (2)
0 Karma
1 Solution

FrankVl
Ultra Champion

You want to change the appearance of the raw events, or you want to extract the fields? Based on your example, I'm assuming the first.

Not entirely sure why you'd want to format the raw events, but something like this should work:

in props.conf

[yoursourcetype]
SEDCMD-0split_to_lines = s/,\s/\n/g
SEDCMD-1strip_quotes = s/"//g
SEDCMD-2add_quotes = s/^([^\r\n]+)/"\1"/

Example in the searchbar:

| makeresults 
| eval _raw = "2018-12-04 01:51:08.330, LogDate=\"2018-12-04 01:51:08.33\", SessionId=\"abc\", MachineName=\"xyz\", LoggerName=\"def\", LogLevel=\"DEBUG\", MessageId=\"DumpCacheNames\", Message=\"def\", ApplicationId=\"fgd\", EndpointStack=\"abc\", LogInsertDate=\"date\""
| rex mode=sed "s/, /\n/g"
| rex mode=sed "s/\"//g"
| rex mode=sed "s/^([^\r\n]+)/\"\1\"/"

View solution in original post

0 Karma

FrankVl
Ultra Champion

You want to change the appearance of the raw events, or you want to extract the fields? Based on your example, I'm assuming the first.

Not entirely sure why you'd want to format the raw events, but something like this should work:

in props.conf

[yoursourcetype]
SEDCMD-0split_to_lines = s/,\s/\n/g
SEDCMD-1strip_quotes = s/"//g
SEDCMD-2add_quotes = s/^([^\r\n]+)/"\1"/

Example in the searchbar:

| makeresults 
| eval _raw = "2018-12-04 01:51:08.330, LogDate=\"2018-12-04 01:51:08.33\", SessionId=\"abc\", MachineName=\"xyz\", LoggerName=\"def\", LogLevel=\"DEBUG\", MessageId=\"DumpCacheNames\", Message=\"def\", ApplicationId=\"fgd\", EndpointStack=\"abc\", LogInsertDate=\"date\""
| rex mode=sed "s/, /\n/g"
| rex mode=sed "s/\"//g"
| rex mode=sed "s/^([^\r\n]+)/\"\1\"/"
0 Karma

vishaltaneja070
Motivator

@FrankVl
Great Buddy. It worked. Thanks 🙂

I want to change the appearance, it is already available in DB connect v1.

0 Karma

493669
Super Champion

Try

...|kv

or in props.conf-

KV_MODE = auto
0 Karma

vishaltaneja070
Motivator

Nup didn't work. Any other suggestion?

0 Karma

493669
Super Champion

i tried with sample data and it is working as expected

| makeresults |eval _raw="\"2018-12-04 01:51:08.330\"
  SessionId=abc 
  MachineName=xyz 
  LoggerName=def
  LogLevel=DEBUG
  MessageId=DumpCacheNames
  Message=def
  ApplicationId=fgd
  EndpointStack=abc
  LogInsertDate=date"| kv
0 Karma

vishaltaneja070
Motivator

Try with this string

2018-12-04 01:51:08.330, LogDate="2018-12-04 01:51:08.33", SessionId="abc", MachineName="xyz", LoggerName="def", LogLevel="DEBUG", MessageId="DumpCacheNames", Message="def", ApplicationId="fgd", EndpointStack="abc", LogInsertDate="date"
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...