Reporting

We have an issue with long JSON log events

bhaskar5428
Explorer

We have an issue with long JSON log events, which is longer than console width limit - they are splitted to 2 separate events, each of them is not a correct JSON. How to handle it correctly? Is it possible to restore broken messages on splunk side, or we need to reach logger to know about width limitation and chunk messages in a proper way? How to handle large JSON events?

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

What do you mean by "console width limit"? If an event is split into two separate ones it's either because it's split before it reaches Splunk or it hits the LINE_BREAKER for give sourcetype. If the event was too long it'd simply get truncated, not split.

And no, you can't join two separate events in Splunk - each event is processed as separate entity (in fact with distributed environment each of those events could end up on a different indexer).

0 Karma

marnall
Builder

Ideally you'd be able to chunk the Json log event into smaller subunits, but this depends on what your JSON log event looks like.

If your json log events are over 10k characters long, they may be getting truncated. If this is the case, you can override the truncation by putting the following setting in a props.conf file on the indexing machines:

[<yoursourcetype>]
TRUNCATE = <some number above the size of your json logs, or 0 for no truncation>

If your broken json logs in Splunk are less than 10k characters long, then it could be that Splunk is splitting the logs part-way through the json object, so you would need to set the LINE_BREAKER field so that it properly splits whole json objects.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...