Splunk Search

I would like to index and make the KV that are in the JSON available as searchable fields

andrewbeak
Path Finder

I am using Splunk Cloud which means I do not have access to the server.

I have log lines that look like this:

Jan  2 18:32:25 hostname analyticsEvent[32152]: {"message":"{"userId":"72",},"extra":[]}

In other words, there is a JSON encoded string in the line.

I would like to index and make the KV that are in the JSON available as searchable fields.

How is this possible?

0 Karma
1 Solution

micahkemp
Champion

To remove non-json from your events, so that the JSON is automatically handled at search type, put this in props.conf:

[<sourcetype name>]
SEDCMD-keepjson = s/^[^{]*{/{/

Note: this will alter _raw at index time. So be sure to test this in a dev system before making this change in production.

View solution in original post

micahkemp
Champion

To remove non-json from your events, so that the JSON is automatically handled at search type, put this in props.conf:

[<sourcetype name>]
SEDCMD-keepjson = s/^[^{]*{/{/

Note: this will alter _raw at index time. So be sure to test this in a dev system before making this change in production.

andrewbeak
Path Finder

Thanks, this looks like what I need. I'm using Splunk Cloud and from what I read the props.conf is set up on the server (which I don't have access to). Do you know how to change it on Splunk Cloud?

0 Karma

micahkemp
Champion

Based on previous answers post:

<your base search>
 | rex "(?<json>\{.+)" 
 | spath input=json

andrewbeak
Path Finder

Thanks for the answer, but I'm hoping to have the keys indexed so that they appear on the left side of the search panel to allow filtering and reporting on their values. I'd prefer to avoid having to use search modifiers as much as possible (not all of my users are programmers). Is there no way to edit the sourcetype to tell it "ignore this pattern of characters and then json_decode the rest"

0 Karma

micahkemp
Champion

You could rewrite _raw at index time. That would happen after timestamping, so it might be a viable solution for you. Your props.conf might look like:

[<sourcetype>]
SEDCMD-keepjson = s/^[^{]*//

Edit: fixed SEDCMD line. Added answer with this info, since it's a different solution than the answer of which this comment is a child.

0 Karma

micahkemp
Champion

I'm refining this response. It doesn't appear to work as currently stated.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...