Getting Data In

Epoch Time

hmrabet2
Observer

Im onboarding sample logs from a txt file to my local Splunk instance were the time stamp is in a 10 digit format (epoch time format). During the onboarding im applying the following timestamp format  strptime("timestamp","%m/%d/%y %H:%M:%S") "timestamp" being the field name in the raw sample in the txt document.  But the timestamp is still defaulting to modtime. Any ideas? 

 

Labels (1)
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

TIME_FORMAT=%s is the proper way to configure a timestamp in epoch format. If your logs are formatted such that Splunk cannot clearly identify which 10-digit value represents a timestamp, you may need to provide more hints (recommended to be explicit anyways), like TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD etc.

If you are able to provide a sample log event, it will be easier to help with more details.

0 Karma

hmrabet2
Observer

Example timestamp in raw logs:        timestamp1617865161

0 Karma

s2_splunk
Splunk Employee
Splunk Employee
TIME_PREFIX=timestamp:\s
TIME_FORMAT=%s

should extract the timestamp properly

0 Karma

hmrabet2
Observer

Thanks, i have added the below to the advanced  section under timestamp but its still defaulting back to modtime. 

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Please share one full event here for further help. You can anonymize data as needed, but please maintain the format of the event.

0 Karma

hmrabet2
Observer

Anonymised raw sample: 

{"hostname":"ip-xxx-xxx-xxx-xx.eu-west-1.compute.internal","query":"xxxxx.net.","response_code":"NXDOMAIN","size":"89","src_ip":"xx.xxx.xxx.xxx","timestamp":"1617865214"}

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

OK, it's JSON format, that's helpful to know....

[yourSourcetypeName]
SHOULD_LINEMERGE=false
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT = %s

 You may have to add other settings here depending on other requirements, like line breakers etc., but this should parse your epoch timestamp as expected.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

strptime is parsing the timestamp field and expecting it to be in the given format, but you have already said it is a 10 digit number (not the format you are trying to parse with)

0 Karma
Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...