Getting Data In

How do you parse multiline key value events?

vishaltaneja070
Motivator

How do you parse the below events?

The events looks like :

2018-12-04 01:51:08.330, LogDate="2018-12-04 01:51:08.33", SessionId="abc", MachineName="xyz", LoggerName="def", LogLevel="DEBUG", MessageId="DumpCacheNames", Message="def", ApplicationId="fgd", EndpointStack="abc", LogInsertDate="date"

I need to show events as:

 "2018-12-04 01:51:08.330"
 SessionId=abc 
 MachineName=xyz 
 LoggerName=def
 LogLevel=DEBUG
 MessageId=DumpCacheNames
 Message=def
 ApplicationId=fgd
 EndpointStack=abc
 LogInsertDate=date

Thanks !

Tags (2)
0 Karma
1 Solution

FrankVl
Ultra Champion

You want to change the appearance of the raw events, or you want to extract the fields? Based on your example, I'm assuming the first.

Not entirely sure why you'd want to format the raw events, but something like this should work:

in props.conf

[yoursourcetype]
SEDCMD-0split_to_lines = s/,\s/\n/g
SEDCMD-1strip_quotes = s/"//g
SEDCMD-2add_quotes = s/^([^\r\n]+)/"\1"/

Example in the searchbar:

| makeresults 
| eval _raw = "2018-12-04 01:51:08.330, LogDate=\"2018-12-04 01:51:08.33\", SessionId=\"abc\", MachineName=\"xyz\", LoggerName=\"def\", LogLevel=\"DEBUG\", MessageId=\"DumpCacheNames\", Message=\"def\", ApplicationId=\"fgd\", EndpointStack=\"abc\", LogInsertDate=\"date\""
| rex mode=sed "s/, /\n/g"
| rex mode=sed "s/\"//g"
| rex mode=sed "s/^([^\r\n]+)/\"\1\"/"

View solution in original post

0 Karma

FrankVl
Ultra Champion

You want to change the appearance of the raw events, or you want to extract the fields? Based on your example, I'm assuming the first.

Not entirely sure why you'd want to format the raw events, but something like this should work:

in props.conf

[yoursourcetype]
SEDCMD-0split_to_lines = s/,\s/\n/g
SEDCMD-1strip_quotes = s/"//g
SEDCMD-2add_quotes = s/^([^\r\n]+)/"\1"/

Example in the searchbar:

| makeresults 
| eval _raw = "2018-12-04 01:51:08.330, LogDate=\"2018-12-04 01:51:08.33\", SessionId=\"abc\", MachineName=\"xyz\", LoggerName=\"def\", LogLevel=\"DEBUG\", MessageId=\"DumpCacheNames\", Message=\"def\", ApplicationId=\"fgd\", EndpointStack=\"abc\", LogInsertDate=\"date\""
| rex mode=sed "s/, /\n/g"
| rex mode=sed "s/\"//g"
| rex mode=sed "s/^([^\r\n]+)/\"\1\"/"
0 Karma

vishaltaneja070
Motivator

@FrankVl
Great Buddy. It worked. Thanks 🙂

I want to change the appearance, it is already available in DB connect v1.

0 Karma

493669
Super Champion

Try

...|kv

or in props.conf-

KV_MODE = auto
0 Karma

vishaltaneja070
Motivator

Nup didn't work. Any other suggestion?

0 Karma

493669
Super Champion

i tried with sample data and it is working as expected

| makeresults |eval _raw="\"2018-12-04 01:51:08.330\"
  SessionId=abc 
  MachineName=xyz 
  LoggerName=def
  LogLevel=DEBUG
  MessageId=DumpCacheNames
  Message=def
  ApplicationId=fgd
  EndpointStack=abc
  LogInsertDate=date"| kv
0 Karma

vishaltaneja070
Motivator

Try with this string

2018-12-04 01:51:08.330, LogDate="2018-12-04 01:51:08.33", SessionId="abc", MachineName="xyz", LoggerName="def", LogLevel="DEBUG", MessageId="DumpCacheNames", Message="def", ApplicationId="fgd", EndpointStack="abc", LogInsertDate="date"
0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...