topic Re: How do I get Splunk to recognise epoch time? in Getting Data In

How do I get Splunk to recognise epoch time?

himynamesdave — Mon, 28 Sep 2020 15:17:11 GMT

I have a static JSON file (240k lines) I would like to index. Here's a the format:

{"name":"fuel_level","value":88.260948,"timestamp":1362061287.027000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.035000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.064000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.098000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.110000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.143000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.154000}
{"name":"fuel_level","value":88.260948,"timestamp":1362061287.177000}

In the data preview mode Splunk is assigning the timestamp based on the last time the file was changed, not the timestamp variable of the event.

How can I get Splunk (using 6) to recognise the timestamp of the event?

i've tried playing with the ../props.conf file based on previous answers but have been unsuccessful so far.

Re: How do I get Splunk to recognise epoch time?

jtrucks — Fri, 15 Nov 2013 00:32:44 GMT

Check out props.conf docs and read the Timestamp extraction configuration section.

The likely answer is in this part:

TIME_FORMAT = <strptime-style format>
* Specifies a strptime format string to extract the date.
* strptime is an industry standard for designating time formats.
* For more information on strptime, see "Configure timestamp recognition" in
  the online documentation.
* TIME_FORMAT starts reading after the TIME_PREFIX. If both are specified, the TIME_PREFIX
  regex must match up to and including the character before the TIME_FORMAT date.
* For good results, the <strptime-style format> should describe the day of the year and the
  time of day.
* Defaults to empty.

This references the Configure timestamp recognition docs which should help you tons, especially the Enhanced strptime() support part.

Re: How do I get Splunk to recognise epoch time?

adrianathome — Fri, 15 Nov 2013 01:43:53 GMT

It would be helpful if you post the format that you tried in props.conf.

It should be as follows:

TIME_FORMAT = %s.%6N

Re: How do I get Splunk to recognise epoch time?

himynamesdave — Fri, 15 Nov 2013 15:45:51 GMT

Thanks! This is what I was originally trying, my mistake was not declaring:

TIME_PREFIX="timestamp":