topic Re: Why does an extracted timestamp field show as _raw? in Splunk Search

Why does an extracted timestamp field show as _raw?

mvanberg — Mon, 30 Jan 2017 22:03:55 GMT

I've setup a field extractions with K=V; format and every field is working correctly except for the first field, "timestamp"

Here's the format I'm starting with:

timestamp=1485969522;addr=3232236035;as=192;volume=356;account=1-53abcef

In transforms.conf:

[kv_extraction]
DELIMS = ";", "="

The result:

timestamp:

 timestamp=1485969522;addr=3232236035;as=192;volume=356;account=1-53abcef

(in other words the timestamp field is being extracted as the entire event or _raw)
*Note _time is showing up correctly

addr:
3232236035
(working correctly and shows only the extracted value for all the remaining fields)

Am I doing something wrong here?

P.S.

I tried adding this to props.conf and it did nothing:

TIME_PREFIX= timestamp=

Re: Why does an extracted timestamp field show as _raw?

DalJeanis — Wed, 01 Feb 2017 03:52:46 GMT

This question would be clearer if you showed some actual dummy data rather than the word "value".

Re: Why does an extracted timestamp field show as _raw?

mvanberg — Wed, 01 Feb 2017 17:24:09 GMT

I added some real data, maybe that will help.

Re: Why does an extracted timestamp field show as _raw?

somesoni2 — Wed, 01 Feb 2017 17:27:44 GMT

You've added a transform.conf entry. Have you related it to your sourcetype in your props.conf? FYI, the attribute TIME_PREFIX is used during event processing (timestamp extraction before indexing) and sets the keyword from where the timestamp is available in _raw and which should be used as _time. It's doesn't help with field extraction.

Re: Why does an extracted timestamp field show as _raw?

mvanberg — Wed, 01 Feb 2017 18:01:58 GMT

I do have the entry added in pops.conf. It's good to know that TIME_PREFIX is done before indexing, because this is all stuff I'm adding to the search heads. It still doesn't explain why the other fields are extracting just fine and this one is ignoring the the delimiters. My guess is that it's because you cannot extract data with the key of "timestamp..." but I have not confirmed this. That, or maybe the first field of an event gets treated differently...

Re: Why does an extracted timestamp field show as _raw?

somesoni2 — Wed, 01 Feb 2017 18:53:02 GMT

Can you try with keeping KV_MODE=none in your props.conf on Search Head? This link explains the order of search time field extractions.
http://docs.splunk.com/Documentation/Splunk/6.5.0/Knowledge/Searchtimeoperationssequence

So, your transform.conf entry (REPORT) gets executed first and creates all fields correctly including timestamp. Then the fields are extracted based on KV_MODE (default to auto), in which timestamp is extracted again and overwrites the current value. It captures whole values as there are no spaces.

Re: Why does an extracted timestamp field show as _raw?

mvanberg — Thu, 02 Feb 2017 18:17:14 GMT

Thanks for the post. I just added "KV_MODE = none" to props.conf and nothing has changed. I even restarted splunk just in case -- though I shouldn't have had to -- and nothing... Any other thoughts?

Re: Why does an extracted timestamp field show as _raw?

somesoni2 — Thu, 02 Feb 2017 18:49:24 GMT

I thought that would be it. Just to confirm, we set KV_MODE = none on search head, under the same sourcetype stanza. Changing the configuration files from the file system would require a restart.