Getting Data In

How to write regex to event break a multi line file into single event?

Hemnaath
Motivator

I have multi line file (_json), which I am trying to create a individual events, the multi line file contains array of id, message and timestamp.

Sample Event data: 

{ [-]
logEvents: [ [-]
{ [-]
id: 3576745055635743000077342515139507954347666517578940416
message: START RequestId: 4e1251df-11d9-55d0-918a-09bb06b96122 Version: $LATEST

timestamp: 1603867953198
}
{ [+]
}
{ [-]
id: 35767450557316368740614159310005543840071546062336098306
message: [2020-10-28T06:52:33.240Z][4e1251df-11d9-55d0-918c-09cc06b96122][INFO][wfm-test2-lmd-towSyncWorkOrderWOM][HeaderProcessor.py, 23][The filtered request headers are {"test-PartyID": "test"}]

timestamp: 1603867953241
}
{ [+]
}
{ [-]
id: 3576745057558067905821073966314329716666554135734059012
message: [2020-10-28T06:52:34.59Z][4e1251df-11d9-55d0-918c-09cc06b96122][INFO][wfm-test2-lmd-towSyncWorkOrderWOM][lambda_function.py, 37][Response received from SNOW with status code :202 and response as {"result":{"message":"Message has been received!","value":"WOR200033942808"}}]

timestamp: 1603867954060
}
{ [+]
}
{ [+]
}
]
logGroup: /aws/lambda/wfm-test2-lmd-towSyncWorkOrderWOM
logStream: 2020/10/28/[$LATEST]0e5e38b8bf8e4247a5f063e5e1fdaf51
messageType: DATA_MESSAGE
owner: 126208963777
subscriptionFilters: [ [+]
]

Can you please guide me how to break this multi line event using the line breaker.

Labels (2)

Hemnaath
Motivator

Hey I was able to break the multi line events into single events using the below stanza 

SHOULD_LINEMERGE=false
LINE_BREAKER=(\[|,\s*|\], )({"id":|"logGroup":)
disabled=false
MAX_TIMESTAMP_LOOKAHEAD=13
TIME_FORMAT=%s%3Q
TIME_PREFIX="timestamp":\s+
TZ=UTC
TRUNCATE=100000

Hemnaath
Motivator

hey tried the same but that is not working 

SHOULD_LINEMERGE = false

LINE_BREAKER=([\r\n]+)\}()\{id:

 

richgalloway
SplunkTrust
SplunkTrust

Have you tried 

LINE_BREAKER = \}()\{id:
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...