Hey Ninjas,
I'm getting the below-parsing error when indexing the JSON formatted events.
ERROR LineBreakingProcessor - Line breaking regex has no capturing groups: \"}
Below are the sample JSON events.
{"timestamp":"2018-06-06T19:13:28.459+00:00","message":"Response time for Mongo query {\"method\":\"find\",\"collection\":\"XXXXXXX\",\"query\":{\"$and\":[{\"region.isoCountryCode\":\"GB\"},{\"abc.branchNumber\":5184},{\"classification.category\":{\"$in\":[\"Store\"]}}],\"status.currentStatus\":{\"$ne\":\"Hidden\"}},\"fields\":{}} with offset: 0, limit: 10, fields: {} is 9 ms","className":"XXXXXXXXXX","threadName":"XXXXXXXX","level":"INFO","HOSTNAME":"ip-1-0-0-0","traceId":"XXXXxXXX"}
{"timestamp":"2018-06-05T14:46:53.919+00:00","message":"Response time for request: 2 ms","className":"XXXXXX","threadName":"XXXXXX","level":"INFO","HOSTNAME":"ip-0-0-0-0","traceId":"xxxxxx"}
I have already tried with below props configuration but no luck.
[test1]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=json
KV_MODE=none
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldown_type=true
[test2]
BREAK_ONLY_BEFORE=^{
CHARSET=UTF-8
SHOULD_LINEMERGE=true
category=Structured
description=A variant of the JSON source type, with support for nonexistent timestamps
disabled=false
pulldown_type=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX=^{\"timestamp\":\"
[test3]
BREAK_ONLY_BEFORE=^{
CHARSET=UTF-8
SHOULD_LINEMERGE=true
category=Structured
description=A variant of the JSON source type, with support for nonexistent timestamps
disabled=false
pulldown_type=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX=^{\"timestamp\":\"
LINE_BREAKER=\"traceId\":\"[\w\d-]+\"}
Please let me know anyone come across on this issue .....
You seem to be setting SHOULD_LINEMERGE
to true
, but your data is one-line/JSON. So you should set it to false
if it is really only one-line per event. Then there are things that your props.conf
won't need to have anymore.
BREAK_ONLY_BEFORE
doesn't need the capture group that the error is complaining about, but the LINE_BREAKER
should have a capture group. But if you take the advice above about the SHOULD_LINEMERGE
, then you shouldn't need LINE_BREAKER
, thus eliminating that error.
Then your TIME_FORMAT
should probably include the TZ info with %:z
.
I would use the following in your props.conf
file:
[ <your-source-type-name-should-be-here. ]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
TIME_FORMAT=%FT%T.%N%:z
TIME_PREFIX="timestamp":"
MAX_TIMESTAMP_LOOKAHEAD=32
That should work an not give you any error messages.
If we use this props then auto field extraction will not work while indexing.
If you are referring to the json formatting and field extraction, just add:
INDEXED_EXTRACTIONS=json
KV_MODE=none