topic Re: How to expand the values as separate events in my JSON data? in Getting Data In

How to expand the values as separate events in my JSON data?

Nadhiyaa — Wed, 29 Aug 2018 06:01:43 GMT

{
    "results": [
        {
            "statement_id": 0,
            "series": [
                {
                    "name": "sqlserver_server_properties",
                    "columns": [
                        "time",
                        "last"
                    ],
                    "values": [
                        [
                            "2018-08-07T00:00:00Z",
                            144
                        ]}

This is my json data . I extracted the time and last using index=..|spath output=time path=results{}.series{}.values{}{0}|spath output=count1 path=results{}.series{}.values{}{1}

I want to expand the values as separate events.

Help would be appreciated.

Re: How to expand the values as separate events in my JSON data?

niketn — Wed, 29 Aug 2018 06:33:23 GMT

@Nadhiyaa based on sample JSON data (I have modified a bit to add ] to format as valid JSON), please try the following run anywhere example:

| makeresults 
| fields - _time 
| eval _raw="{
\"results\": [
{
\"statement_id\": 0,
\"series\": [
{
\"name\": \"sqlserver_server_properties\",
\"columns\": [
\"time\",
\"last\"
    ],
\"values\": [
[
\"2018-08-07T00:00:00Z\",
144
]]}]}]}"
| spath 
| rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
| eval data=mvzip(columns, values) 
| mvexpand data 
| makemv data delim="," 
| eval columns=mvindex(data,0),values=mvindex(data,1) 
| fields - data

PS: command from | makeresults till | spath generates dummy data and fields. You should define KV_MODE=json in the props.conf to have all the JSON fields automatically extracted during search time.
The mvzip command is used to bring multivalued fields together (unless there is one to one mapping in the columns and values fields, this approach will not work and dummy value needs to be inserted at search time or index time (from source if possible).
Then mvexpand splits them in two rows. Finally, makemv and eval with mvindex() is used to get the original columns and values field values back.

Re: How to expand the values as separate events in my JSON data?

Nadhiyaa — Wed, 29 Aug 2018 07:24:16 GMT

Hi @niketnilay

I validated the json and provided kvmode as json.

got the ouput below using
|spath output=time path=results{}.series{}.values{}{0}|spath output=last path=results{}.series{}.values{}{1}
|eval time=strptime(time,"%Y-%m-%dT%H:%M:%SZ")|search count1!=null|table

Its coming as multivalue fields i want to have one to one in the separate rows.

Re: How to expand the values as separate events in my JSON data?

Nadhiyaa — Wed, 29 Aug 2018 09:53:26 GMT

@niketnilay How to avoid dummy data and fields .however i defined KV_MODE=json in the props.conf.Does this avoid dummy data

Re: How to expand the values as separate events in my JSON data?

niketn — Wed, 29 Aug 2018 10:27:44 GMT

@Nadhiyaa what do you mean by dummy data!If that is with respect to my query, you just need to remove first two pipes i.e. makeresults and eval _raw from the sample query and place your main search instead i.e. index=..

  index=.....
 | rename results{}.series{}.*{} as *, results{}.series{}.*{}{} as * 
 | eval data=mvzip(columns, values) 
 | mvexpand data 
 | makemv data delim="," 
 | eval columns=mvindex(data,0),values=mvindex(data,1) 
 | fields - data