How to automatically extract the JSON object before indexing so I don't have to use spath in my search?

[Kukkadapu]

Hi,

How do I extract the JSON object before indexing itself? Right now I'm extracting using the below search.

This is the data:

2016-01-18 16:24:40,406 INFO  [org.apache.log4j.Logger] (ajp-/10.32.20.21:8309-7) 
transaction_id="123451" 
action="ABC API" 
desc="start of api" 
result="success" 
http_method="POST"
payload_json=
{
    "requestId": "ABCDEqq",
    "partnerId": "001",
    "storeId": "001",
    "subscriberId": "001",
    "event": "1",
    "date": "2015-10-20 12:08:56 PDT",
    "uuid": "123451"
}

Here is the search:

index="xyz" sourcetype="pm" action="ABC API" | spath input=payload_json  | stats count by action,event

It works fine, but is there a way to extract the JSON before indexing itself so the search is going to be:

index="xyz" sourcetype="pm" action="ABC API" | stats count by action,event 

No spath in this command

[ssievert_splunk · Accepted solution]

If your payload_json is pretty static, you could create calculated fields using spath(payload_json, "requestId") (for example) as the eval expression. It would still execute at search time, though. There probably is a way to write a generic EXTRACT/TRANSFORM as well to dynamically get the fields extracted from the payload_json field you already have.