Using Kinesis, AWS sends multiple messages in a single JSON body. How to event break JSON messages without using "mvexpand" command?
When we use Kinesis, AWS is sending multiple messages in a single JSON body to Splunk. I need help with breaking the messages in JSON body of an event and create separate events for each message.
Here is the search:
index=aws_xxx sourcetype="aws:kinesis-*-kinesis-*" | table _time, logEvents{}.message
When I run this search, it shows multiple records of logEvents{}.message per event. What I need is a single row for each message.
New search to fix this issue with mvexpand
command:
index=aws_xxx sourcetype="aws:kinesis-*-kinesis-*" | mvexpand logEvents{}.message | table _time, logEvents{}.message
The above works fine but the mvexpand
command has limits of 6500 records which can be updated with limits.conf and other parameters but it will impact the Indexer RAM drastically and I feel that is not a performant way to resolve this issue. Instead I am looking for any substitute of the mvexpand
command or trying to find if there is a simpler way to break the events for JSON body so that each message is tagged as a new line within Splunk (may be using props.conf).
I am also having the same issue where AWS is sending multiple messages in a single JSON body to Splunk. our Splunk Add-on for AWS is 4.0.0; I wanted to know if upgrading to 4.4.0 will fix the issue. Also, will I be able to easily downgrade from 4.4.0 to 4.0.0 if needed? Thanks
Hi ash21,
Which version of AWS Add-on are you using to collect Kinesis data? The latest release is 4.2.1 available on SplunkBase:
http://splunkbase.splunk.com/app/1876
Please make sure you configure the Kinesis input correctly and the Splunk Add-on for AWS will ingest Kinesis events for you without the need to customize any configurations:
http://docs.splunk.com/Documentation/AddOns/released/AWS/Kinesis
Hope this helps. Thanks!
Hunter
Hi,
We are having a similar issue where AWS is sending multiple messages in a single JSON body to Splunk. We are currently on splunk 4.0.0; I wanted to know if upgrading to splunk 4.4.0 will fix the issue?
if upgrading to splunk 4.4.0 will indeed fix the issue, and we choose to upgrade and upgrading causes other unforeseen issues, will it be fairly easy to downgrade? Can we easily downgrade back to 4.4.0 ourselves?
The version of Splunk Add-on for AWS is 4.0.0
Is this specific issue fixed in 4.2.1?
Kinesis output is correctly configured based on the document. We see data coming in from Kinesis stream however the JSON events are embedded with multiple messages in it.
The workaround we currently have to break the events by messages within JSON body is:
index=aws_xxx sourcetype="aws:kinesis--kinesis-" | stats count by _time, logEvents{}.message | table _time, logEvents{}.message