Splunk Search

Can Splunk dynamically extract JSON Key names while joining the event with other sourcelogs?

lpolo
Motivator

The following query is able to join two source logs where one of the source logs is in json format:

(sourcetype="request" AND application=vsp NOT (Agent.007) key_name1 )
 OR
(sourcetype="response" key_name2)
| spath
| spath path=your_json_path output=your_output_key_name1
| spath path=your_json_path output=your_output_key_name2
| spath path=your_json_path output=your_output_key_name3
...
| spath path=your_json_path output=your_output_key_name4
stats
 first(your_output_key_name1) as your_output_key_name1
 first(your_output_key_name2) as your_output_key_name2
 first(your_output_key_name3) as your_output_key_name3
 first(your_output_key_name4) as your_output_key_name4
 first(key_name1) as key1
 list(key_name2) as key2 
 dc(sourcetype) as dc by id

Problem:
JSon path could vary. Therefore, output variables too. Is there a way in Splunk that these could be discovered dynamically?

Regards,
Lp

Tags (3)
0 Karma

rgcurry
Contributor

Have you considered using the "KV_MODE" parameter in your props.conf for this sourcetype? Check it out at http://docs.splunk.com/Documentation/Splunk/5.0.2/Knowledge/Createandmaintainsearch-timefieldextract....

0 Karma

lpolo
Motivator

It could be an option too... I have to test it.

0 Karma

lpolo
Motivator

This query works:

(sourcetype="request")//sourcelog. Key=value pair format.
 OR
(sourcetype="json_response")//source og. json format.
  |spath  //spath command in auto-extract mode. 
  |stats list(*) as * by id //join source log by id.

However, I am not source if it is limited by the way spath is ran without arguments. As documented:

When spath is ran with no path argument, spath runs in "auto-extract" mode, where it finds and extracts all the fields from the first 5000 characters in the input field (which defaults to _raw if another input source isn't specified).
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Meet the .conf26 AI Agent Buildathon Winners

At the .conf26 AI Agent Buildathon, Splunk users turned their operational expertise into working AI agents ...

Trusted Autonomy in the SOC: A Maturity Journey

AI is compressing attacker timelines and pushing SOC teams beyond human capacity thresholds. The next SOC ...

Supercharging Windows Security Detection Performance: Introducing Hybrid Field ...

Windows event logs—from Security auditing and Sysmon to PowerShell script blocks—form the operational backbone ...