Getting Data In

Read only the JSON section of each line in a monitored file

jeffbat
Path Finder

I am working on seeing if there is a better way that I can consume the data I have from an Azure Blob storage.

I am using the Splunk Addon for Microsoft Cloud Services; which is allowing me to grab the file from the Blob storage.

Now the file itself is actually a CSV; but the app does not recognize CSV to split up the data so I had to create a props.conf and transforms.conf which does allow me to get the data in and into fields.  But it turns out that the CSV fields has 1 field in it that is the rawMessage and it is actually a JSON of all the field data (and sometimes will contain a field which is not broken into a column of the CSV. 

For now I am bringing all of the data in and separated into fields and then at search time doing a spath on the rawMessage field to get the other fields that only sometimes appear.

What I would like to do is instead of all of the data being brought in; I want to only bring in the rawMessage field (which is JSON) and have that indexed as it has all of the data/fields.

What is the best way to write the inputs/props/transforms to only read that field in and then parse it as a JSON so it will do the autobreakdown for fields?

Any help with this would be greatly appreciated.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...