Structured fields and values in json api results

[stevesq]

In your REST API documentation you have the following json example:

    // sample JSON output
// https://localhost:8089/services/search/jobs/1234/results?output_mode=json

[
{
"_cd": "0:4374557",
"_index": "main",
"_kv": "1",
"_meta": " date_second::36 date_hour::19 date_minute::11 date_year::2008 date_month::january date_mday::21 date_wday::monday date_zone::-480 punct::_[//:::_-]____\\\"@...\\\"...",
"_raw": "I [21/Jan/2008:19:11:36 -0800] Added remote printer \"HPLaserJ@10.1.1.123\"...",
"_serial": "0",
"_time": "1200971496",
"date_hour": "19",
"date_mday": "21",
"date_minute": "11",
"date_month": "january",
"date_second": "36",
"date_wday": "monday",
"date_year": "2008",
"date_zone": "-480",
"host": "decider.local",
"linecount": "1",
"punct": "_[//:::_-]____\"@...\"...",
"source": "/var/log/cups/error_log",
"sourcetype": "cups_error"
},

However when I make an api request and then check results from the /results endpoint, I only see the raw data and a few other fields, like so:

{
"_cd": "6:719660",
"_indextime": "1305696946",
"_raw": "id=\"123\" color=\"red\" foo= model=\"accord\"",
"_serial": "93",
"_si": "log.example.com\nmain",
"_sourcetype": "car_sourcetype",
"_time": "2010-07-18T19:19:30.000+00:00",
"host": "log.example.com",
"index": "main",
"linecount": "1",
"source": "cars",
"sourcetype": "cars_sourcetype",
"splunk_server": "log.example.com"

},

Note that _kv is missing, as well as the broken-out keys and values. What triggers these to be put in the api results? I can't find anything obvious in the api docs.

-Steve

[stevesq · Accepted solution]

I made it work by including

required_field_list=*

in the search POST:

curl -u admin:foobar -k https://localhost:8089/services/search/jobs -d"search=search source=cars | head 1 &required_field_list=*"

Could be a bug.

[hazekamp]

stevesq,

I suspect key-value pair extracting is not occurring for your sourcetype. Key-value extraction is configured via props.conf/transforms.conf. Once key-value pair extractions are specified for your data, search results will reflect this information and so will results retrieved via the api.

What's interesting is that Splunk is not automatically extracting key=value pairs in your _raw message. Perhaps KV_MODE is set to None or "Field discovery" was disabled (if search was executed from the UI).

Can you verify flashtimeline searches return the expected "id/color/foo/model" fields?

[stevesq]

"search results will reflect this information and so will results retrieved via the api."

My search results seem to be keyed fine - e.g. if I pipe a search to "table *" or search for "color=red", the right things happen. Splunk is clearly treating my data as properly separated-out rows/cols/cells.