Getting Data In

How to expand the values as separate events in my JSON data?

Nadhiyaa
Path Finder
{
    "results": [
        {
            "statement_id": 0,
            "series": [
                {
                    "name": "sqlserver_server_properties",
                    "columns": [
                        "time",
                        "last"
                    ],
                    "values": [
                        [
                            "2018-08-07T00:00:00Z",
                            144
                        ]}

This is my json data . I extracted the time and last using index=..|spath output=time path=results{}.series{}.values{}{0}|spath output=count1 path=results{}.series{}.values{}{1}

I want to expand the values as separate events.

Help would be appreciated.

0 Karma
1 Solution

@Nadhiyaa based on sample JSON data (I have modified a bit to add ] to format as valid JSON), please try the following run anywhere example:

| makeresults 
| fields - _time 
| eval _raw="{
\"results\": [
{
\"statement_id\": 0,
\"series\": [
{
\"name\": \"sqlserver_server_properties\",
\"columns\": [
\"time\",
\"last\"
    ],
\"values\": [
[
\"2018-08-07T00:00:00Z\",
144
]]}]}]}"
| spath 
| rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
| eval data=mvzip(columns, values) 
| mvexpand data 
| makemv data delim="," 
| eval columns=mvindex(data,0),values=mvindex(data,1) 
| fields - data

PS: command from | makeresults till | spath generates dummy data and fields. You should define KV_MODE=json in the props.conf to have all the JSON fields automatically extracted during search time.
The mvzip command is used to bring multivalued fields together (unless there is one to one mapping in the columns and values fields, this approach will not work and dummy value needs to be inserted at search time or index time (from source if possible).
Then mvexpand splits them in two rows. Finally, makemv and eval with mvindex() is used to get the original columns and values field values back.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

@Nadhiyaa based on sample JSON data (I have modified a bit to add ] to format as valid JSON), please try the following run anywhere example:

| makeresults 
| fields - _time 
| eval _raw="{
\"results\": [
{
\"statement_id\": 0,
\"series\": [
{
\"name\": \"sqlserver_server_properties\",
\"columns\": [
\"time\",
\"last\"
    ],
\"values\": [
[
\"2018-08-07T00:00:00Z\",
144
]]}]}]}"
| spath 
| rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
| eval data=mvzip(columns, values) 
| mvexpand data 
| makemv data delim="," 
| eval columns=mvindex(data,0),values=mvindex(data,1) 
| fields - data

PS: command from | makeresults till | spath generates dummy data and fields. You should define KV_MODE=json in the props.conf to have all the JSON fields automatically extracted during search time.
The mvzip command is used to bring multivalued fields together (unless there is one to one mapping in the columns and values fields, this approach will not work and dummy value needs to be inserted at search time or index time (from source if possible).
Then mvexpand splits them in two rows. Finally, makemv and eval with mvindex() is used to get the original columns and values field values back.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

Nadhiyaa
Path Finder

@niketnilay How to avoid dummy data and fields .however i defined KV_MODE=json in the props.conf.Does this avoid dummy data

0 Karma

@Nadhiyaa what do you mean by dummy data!If that is with respect to my query, you just need to remove first two pipes i.e. makeresults and eval _raw from the sample query and place your main search instead i.e. index=..

  index=.....
 | rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
 | eval data=mvzip(columns, values) 
 | mvexpand data 
 | makemv data delim="," 
 | eval columns=mvindex(data,0),values=mvindex(data,1) 
 | fields - data
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Nadhiyaa
Path Finder

Hi @niketnilay

I validated the json and provided kvmode as json.

got the ouput below using
|spath output=time path=results{}.series{}.values{}{0}|spath output=last path=results{}.series{}.values{}{1}
|eval time=strptime(time,"%Y-%m-%dT%H:%M:%SZ")|search count1!=null|table

Its coming as multivalue fields i want to have one to one in the separate rows.

0 Karma
Get Updates on the Splunk Community!

This Week's Community Digest - Splunk Community Happenings [9.26.22]

Get the latest news and updates from the Splunk Community here! Upcoming User Group Events! 👏 Check ...

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...