Splunk Search

How do I extract these fields and corresponding values parsing while JSON data?

sarfarajsayyad
New Member

I have a following JSON input.

{
    "StartTime": {
        "@item": "1",
        "#text": "2016/11/21 09:35:25"
    },
    "Encryption": {
        "@item": "1",
        "#text": "None"
    },
    "Duration": {
        "@item": "1",
        "#text": "13"
    },
    "DisplayName": {
        "@item": "1",
        "#text": "80081"
    },
    "device_id": 1
}

I need to extract fields like below and ignore the rest (@item) from JSON.
device_id = 1
DisplayName=80081
Duration=13
Encryption=None
StartTime="2016/11/21 09:35:25"

Can it be done writing a single Regex to extract all the fields, or do I need to write separate regex for each field in TRANSFORMS.CONF ?

Any example will help.
/Sarfaraj

0 Karma

gokadroid
Motivator

Like @Sundaresh mentions to get them extracted automatically, but here are the ways to do this in search time (brute force) if really required

Using spath:

your query to return events
| spath output=device_id path=device_id
| spath output=startTime path=StartTime.#text 
| spath output=encryption path=Encryption.#text 
| spath output=duration path=Duration.#text 
| spath output=displayName path=DisplayName.#text 
| table device_id, startTime, encryption, duration, displayName

Using rex:

your query to return events
| rex field=_raw max_match=0 "\#text\"\:\s\"(?<jsonFields>[^\"]+)"
| rex field=_raw "device_id\"\:\s*(?<device_id>[\S]+)"
| eval startTime=mvindex(jsonFields, 0)
| eval encryption=mvindex(jsonFields, 1)
| eval duration=mvindex(jsonFields, 2)
| eval displayName=mvindex(jsonFields, 3)
| table startTime, encryption, duration, displayName, device_id
0 Karma

sarfarajsayyad
New Member

Thank you ! Is there any way to do it in index time ?

0 Karma

sundareshr
Legend

If it is valid JSON, add this to your props.conf

[ stanza]
KV_MODE = json
0 Karma

sarfarajsayyad
New Member

Dear, I have added that already.
By default its giving me columns like StartTime.@item , StartTime.#text. But i need only StartTime having value of StartTime.#text. Something like StartTime="2016/11/21 09:35:25".

0 Karma

sundareshr
Legend

You can create aliases as suggested of you can rename the fields in your search rename *.#text AS * will rename all <>.#text to <>

0 Karma

Richfez
SplunkTrust
SplunkTrust

If they're already parsed like that, why not just create a simple alias for them, like they explain in this answer?

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...