Splunk Search

How to automatically extract the JSON object before indexing so I don't have to use spath in my search?

Path Finder

Hi,

How do I extract the JSON object before indexing itself? Right now I'm extracting using the below search.

This is the data:

2016-01-18 16:24:40,406 INFO  [org.apache.log4j.Logger] (ajp-/10.32.20.21:8309-7) 
transaction_id="123451" 
action="ABC API" 
desc="start of api" 
result="success" 
http_method="POST"
payload_json=
{
    "requestId": "ABCDEqq",
    "partnerId": "001",
    "storeId": "001",
    "subscriberId": "001",
    "event": "1",
    "date": "2015-10-20 12:08:56 PDT",
    "uuid": "123451"
}

Here is the search:

index="xyz" sourcetype="pm" action="ABC API" | spath input=payload_json  | stats count by action,event

It works fine, but is there a way to extract the JSON before indexing itself so the search is going to be:

index="xyz" sourcetype="pm" action="ABC API" | stats count by action,event 

No spath in this command

0 Karma
1 Solution

Splunk Employee
Splunk Employee

If your payload_json is pretty static, you could create calculated fields using spath(payload_json, "requestId") (for example) as the eval expression. It would still execute at search time, though. There probably is a way to write a generic EXTRACT/TRANSFORM as well to dynamically get the fields extracted from the payload_json field you already have.

View solution in original post

Splunk Employee
Splunk Employee

If your payload_json is pretty static, you could create calculated fields using spath(payload_json, "requestId") (for example) as the eval expression. It would still execute at search time, though. There probably is a way to write a generic EXTRACT/TRANSFORM as well to dynamically get the fields extracted from the payload_json field you already have.

View solution in original post

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!