Splunk Add-On for Microsoft Cloud Services - How to fix Eventhub input parsing error of JSON?

[thilles]

Hi! 

We've set up an Eventhub input using the Splunk Add-on for MS Cloud Services, and we are getting events successfully into Splunk.

The problem is that the events are not formatted correctly when indexed into Splunk. 
JSON formatted events are indexed into Splunk with all the quotation marks escaped. This messes up the syntax highlighting, and the auto field extractions.

The sourcetype used during eventhub config is mscs:azure:eventhub, as per the docs states.

The following code is used to generate test data, and the rendered result is in the screenshot. 

Anyone seen the same, or resolved it somehow?

#Method 1 - You provide a JSON string
body1 = "{'id':'device2','goo':'frodo'}"
event_data_batch.add(EventData(body1))

#Method 2 - You get the JSON Object and convert to string
json_obj = {"id": "device3", "goo": "bilbo"}
body2 = json.dumps(json_obj)
event_data_batch.add(EventData(body2))

#This just sending the string which will not be captured by TSI
event_data_batch.add(EventData('Third event'))

[kristenbidbax · Accepted solution]

add event_format_flags = 1  to your inputs as per documentation
https://docs.splunk.com/Documentation/AddOns/released/MSCloudServices/Upgrade

[thilles]

As an added input:
I've used the Eventhub integration to collect events with proper json rendering earlier, but then the data source was Microsofts own streaming options to Eventhub, e.g.  Stream Microsoft Defender for Endpoint events to Azure Event Hubs | Microsoft Learn