How do I parse JSON events from a custom generating command that queries an API?

[yogevyuval]

Hi,

I have an external API that I want to be able to let my users explore with Splunk.

This API returns a list of deeply nested events in JSON format.

I managed to query the API myself and send the events to Splunk, and this approach works well in terms of indexing of the data.

However, I would like to give my users a faster and more intuitive way of exploring the data so I decided to write a custom Splunk command that would query the API on the fly and let the users search the data — so my command looks something like this:

| <my_custom_command> | stats count by ....

The problem I encountered is that the JSON returned from this generating command is not being parsed well by Splunk. It seems like only the top-level fields have been extracted and their value is just the rest of the JSON.

I understand that a custom command is run after indexing, but this approach of a custom command gives me the flexibility that I want, so I would really be happy to find a way to parse my events successfully.

[martin_mueller · Accepted solution]

Assuming your custom command yields a _raw string containing the json, just like an already-index event would look like, you'll probably want to call the spath command to parse the json string into fields.

[yogevyuval]

Thank you! It worked as expected.