Getting Data In

How to write regex to event break a multi line file into single event?

Hemnaath
Motivator

I have multi line file (_json), which I am trying to create a individual events, the multi line file contains array of id, message and timestamp.

Sample Event data: 

{ [-]
logEvents: [ [-]
{ [-]
id: 3576745055635743000077342515139507954347666517578940416
message: START RequestId: 4e1251df-11d9-55d0-918a-09bb06b96122 Version: $LATEST

timestamp: 1603867953198
}
{ [+]
}
{ [-]
id: 35767450557316368740614159310005543840071546062336098306
message: [2020-10-28T06:52:33.240Z][4e1251df-11d9-55d0-918c-09cc06b96122][INFO][wfm-test2-lmd-towSyncWorkOrderWOM][HeaderProcessor.py, 23][The filtered request headers are {"test-PartyID": "test"}]

timestamp: 1603867953241
}
{ [+]
}
{ [-]
id: 3576745057558067905821073966314329716666554135734059012
message: [2020-10-28T06:52:34.59Z][4e1251df-11d9-55d0-918c-09cc06b96122][INFO][wfm-test2-lmd-towSyncWorkOrderWOM][lambda_function.py, 37][Response received from SNOW with status code :202 and response as {"result":{"message":"Message has been received!","value":"WOR200033942808"}}]

timestamp: 1603867954060
}
{ [+]
}
{ [+]
}
]
logGroup: /aws/lambda/wfm-test2-lmd-towSyncWorkOrderWOM
logStream: 2020/10/28/[$LATEST]0e5e38b8bf8e4247a5f063e5e1fdaf51
messageType: DATA_MESSAGE
owner: 126208963777
subscriptionFilters: [ [+]
]

Can you please guide me how to break this multi line event using the line breaker.

Labels (2)

Hemnaath
Motivator

Hey I was able to break the multi line events into single events using the below stanza 

SHOULD_LINEMERGE=false
LINE_BREAKER=(\[|,\s*|\], )({"id":|"logGroup":)
disabled=false
MAX_TIMESTAMP_LOOKAHEAD=13
TIME_FORMAT=%s%3Q
TIME_PREFIX="timestamp":\s+
TZ=UTC
TRUNCATE=100000

Hemnaath
Motivator

hey tried the same but that is not working 

SHOULD_LINEMERGE = false

LINE_BREAKER=([\r\n]+)\}()\{id:

 

richgalloway
SplunkTrust
SplunkTrust

Have you tried 

LINE_BREAKER = \}()\{id:
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...