Getting Data In

How to expand the values as separate events in my JSON data?

Nadhiyaa
Path Finder
{
    "results": [
        {
            "statement_id": 0,
            "series": [
                {
                    "name": "sqlserver_server_properties",
                    "columns": [
                        "time",
                        "last"
                    ],
                    "values": [
                        [
                            "2018-08-07T00:00:00Z",
                            144
                        ]}

This is my json data . I extracted the time and last using index=..|spath output=time path=results{}.series{}.values{}{0}|spath output=count1 path=results{}.series{}.values{}{1}

I want to expand the values as separate events.

Help would be appreciated.

0 Karma
1 Solution

niketn
Legend

@Nadhiyaa based on sample JSON data (I have modified a bit to add ] to format as valid JSON), please try the following run anywhere example:

| makeresults 
| fields - _time 
| eval _raw="{
\"results\": [
{
\"statement_id\": 0,
\"series\": [
{
\"name\": \"sqlserver_server_properties\",
\"columns\": [
\"time\",
\"last\"
    ],
\"values\": [
[
\"2018-08-07T00:00:00Z\",
144
]]}]}]}"
| spath 
| rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
| eval data=mvzip(columns, values) 
| mvexpand data 
| makemv data delim="," 
| eval columns=mvindex(data,0),values=mvindex(data,1) 
| fields - data

PS: command from | makeresults till | spath generates dummy data and fields. You should define KV_MODE=json in the props.conf to have all the JSON fields automatically extracted during search time.
The mvzip command is used to bring multivalued fields together (unless there is one to one mapping in the columns and values fields, this approach will not work and dummy value needs to be inserted at search time or index time (from source if possible).
Then mvexpand splits them in two rows. Finally, makemv and eval with mvindex() is used to get the original columns and values field values back.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

@Nadhiyaa based on sample JSON data (I have modified a bit to add ] to format as valid JSON), please try the following run anywhere example:

| makeresults 
| fields - _time 
| eval _raw="{
\"results\": [
{
\"statement_id\": 0,
\"series\": [
{
\"name\": \"sqlserver_server_properties\",
\"columns\": [
\"time\",
\"last\"
    ],
\"values\": [
[
\"2018-08-07T00:00:00Z\",
144
]]}]}]}"
| spath 
| rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
| eval data=mvzip(columns, values) 
| mvexpand data 
| makemv data delim="," 
| eval columns=mvindex(data,0),values=mvindex(data,1) 
| fields - data

PS: command from | makeresults till | spath generates dummy data and fields. You should define KV_MODE=json in the props.conf to have all the JSON fields automatically extracted during search time.
The mvzip command is used to bring multivalued fields together (unless there is one to one mapping in the columns and values fields, this approach will not work and dummy value needs to be inserted at search time or index time (from source if possible).
Then mvexpand splits them in two rows. Finally, makemv and eval with mvindex() is used to get the original columns and values field values back.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

Nadhiyaa
Path Finder

@niketnilay How to avoid dummy data and fields .however i defined KV_MODE=json in the props.conf.Does this avoid dummy data

0 Karma

niketn
Legend

@Nadhiyaa what do you mean by dummy data!If that is with respect to my query, you just need to remove first two pipes i.e. makeresults and eval _raw from the sample query and place your main search instead i.e. index=..

  index=.....
 | rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
 | eval data=mvzip(columns, values) 
 | mvexpand data 
 | makemv data delim="," 
 | eval columns=mvindex(data,0),values=mvindex(data,1) 
 | fields - data
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Nadhiyaa
Path Finder

Hi @niketnilay

I validated the json and provided kvmode as json.

got the ouput below using
|spath output=time path=results{}.series{}.values{}{0}|spath output=last path=results{}.series{}.values{}{1}
|eval time=strptime(time,"%Y-%m-%dT%H:%M:%SZ")|search count1!=null|table

Its coming as multivalue fields i want to have one to one in the separate rows.

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...