Getting Data In

How to write regex to event break a multi line file into single event?

Hemnaath
Motivator

I have multi line file (_json), which I am trying to create a individual events, the multi line file contains array of id, message and timestamp.

Sample Event data: 

{ [-]
logEvents: [ [-]
{ [-]
id: 3576745055635743000077342515139507954347666517578940416
message: START RequestId: 4e1251df-11d9-55d0-918a-09bb06b96122 Version: $LATEST

timestamp: 1603867953198
}
{ [+]
}
{ [-]
id: 35767450557316368740614159310005543840071546062336098306
message: [2020-10-28T06:52:33.240Z][4e1251df-11d9-55d0-918c-09cc06b96122][INFO][wfm-test2-lmd-towSyncWorkOrderWOM][HeaderProcessor.py, 23][The filtered request headers are {"test-PartyID": "test"}]

timestamp: 1603867953241
}
{ [+]
}
{ [-]
id: 3576745057558067905821073966314329716666554135734059012
message: [2020-10-28T06:52:34.59Z][4e1251df-11d9-55d0-918c-09cc06b96122][INFO][wfm-test2-lmd-towSyncWorkOrderWOM][lambda_function.py, 37][Response received from SNOW with status code :202 and response as {"result":{"message":"Message has been received!","value":"WOR200033942808"}}]

timestamp: 1603867954060
}
{ [+]
}
{ [+]
}
]
logGroup: /aws/lambda/wfm-test2-lmd-towSyncWorkOrderWOM
logStream: 2020/10/28/[$LATEST]0e5e38b8bf8e4247a5f063e5e1fdaf51
messageType: DATA_MESSAGE
owner: 126208963777
subscriptionFilters: [ [+]
]

Can you please guide me how to break this multi line event using the line breaker.

Labels (2)

Hemnaath
Motivator

Hey I was able to break the multi line events into single events using the below stanza 

SHOULD_LINEMERGE=false
LINE_BREAKER=(\[|,\s*|\], )({"id":|"logGroup":)
disabled=false
MAX_TIMESTAMP_LOOKAHEAD=13
TIME_FORMAT=%s%3Q
TIME_PREFIX="timestamp":\s+
TZ=UTC
TRUNCATE=100000

Hemnaath
Motivator

hey tried the same but that is not working 

SHOULD_LINEMERGE = false

LINE_BREAKER=([\r\n]+)\}()\{id:

 

richgalloway
SplunkTrust
SplunkTrust

Have you tried 

LINE_BREAKER = \}()\{id:
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...