topic Re: Epoch Time in Getting Data In

Epoch Time

hmrabet2 — Thu, 29 Apr 2021 13:44:15 GMT

Im onboarding sample logs from a txt file to my local Splunk instance were the time stamp is in a 10 digit format (epoch time format). During the onboarding im applying the following timestamp format strptime("timestamp","%m/%d/%y %H:%M:%S") "timestamp" being the field name in the raw sample in the txt document. But the timestamp is still defaulting to modtime. Any ideas?

Re: Epoch Time

ITWhisperer — Thu, 29 Apr 2021 14:40:15 GMT

strptime is parsing the timestamp field and expecting it to be in the given format, but you have already said it is a 10 digit number (not the format you are trying to parse with)

Re: Epoch Time

s2_splunk — Thu, 29 Apr 2021 19:36:47 GMT

TIME_FORMAT=%s is the proper way to configure a timestamp in epoch format. If your logs are formatted such that Splunk cannot clearly identify which 10-digit value represents a timestamp, you may need to provide more hints (recommended to be explicit anyways), like TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD etc.

If you are able to provide a sample log event, it will be easier to help with more details.

Re: Epoch Time

hmrabet2 — Fri, 30 Apr 2021 09:14:04 GMT

Example timestamp in raw logs: timestamp: 1617865161

Re: Epoch Time

s2_splunk — Fri, 30 Apr 2021 16:40:56 GMT

TIME_PREFIX=timestamp:\s TIME_FORMAT=%s

should extract the timestamp properly

Re: Epoch Time

hmrabet2 — Fri, 30 Apr 2021 16:50:30 GMT

Thanks, i have added the below to the advanced section under timestamp but its still defaulting back to modtime.

Re: Epoch Time

s2_splunk — Fri, 30 Apr 2021 17:09:36 GMT

Please share one full event here for further help. You can anonymize data as needed, but please maintain the format of the event.

Re: Epoch Time

hmrabet2 — Tue, 04 May 2021 08:30:16 GMT

Anonymised raw sample:

{"hostname":"ip-xxx-xxx-xxx-xx.eu-west-1.compute.internal","query":"xxxxx.net.","response_code":"NXDOMAIN","size":"89","src_ip":"xx.xxx.xxx.xxx","timestamp":"1617865214"}

Re: Epoch Time

s2_splunk — Tue, 04 May 2021 15:55:09 GMT

OK, it's JSON format, that's helpful to know....

[yourSourcetypeName] SHOULD_LINEMERGE=false INDEXED_EXTRACTIONS = json KV_MODE = none TIMESTAMP_FIELDS = timestamp TIME_FORMAT = %s

You may have to add other settings here depending on other requirements, like line breakers etc., but this should parse your epoch timestamp as expected.