Getting Data In

Epoch Time

hmrabet2
Observer

Im onboarding sample logs from a txt file to my local Splunk instance were the time stamp is in a 10 digit format (epoch time format). During the onboarding im applying the following timestamp format  strptime("timestamp","%m/%d/%y %H:%M:%S") "timestamp" being the field name in the raw sample in the txt document.  But the timestamp is still defaulting to modtime. Any ideas? 

 

Labels (1)
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

TIME_FORMAT=%s is the proper way to configure a timestamp in epoch format. If your logs are formatted such that Splunk cannot clearly identify which 10-digit value represents a timestamp, you may need to provide more hints (recommended to be explicit anyways), like TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD etc.

If you are able to provide a sample log event, it will be easier to help with more details.

0 Karma

hmrabet2
Observer

Example timestamp in raw logs:        timestamp1617865161

0 Karma

s2_splunk
Splunk Employee
Splunk Employee
TIME_PREFIX=timestamp:\s
TIME_FORMAT=%s

should extract the timestamp properly

0 Karma

hmrabet2
Observer

Thanks, i have added the below to the advanced  section under timestamp but its still defaulting back to modtime. 

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Please share one full event here for further help. You can anonymize data as needed, but please maintain the format of the event.

0 Karma

hmrabet2
Observer

Anonymised raw sample: 

{"hostname":"ip-xxx-xxx-xxx-xx.eu-west-1.compute.internal","query":"xxxxx.net.","response_code":"NXDOMAIN","size":"89","src_ip":"xx.xxx.xxx.xxx","timestamp":"1617865214"}

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

OK, it's JSON format, that's helpful to know....

[yourSourcetypeName]
SHOULD_LINEMERGE=false
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT = %s

 You may have to add other settings here depending on other requirements, like line breakers etc., but this should parse your epoch timestamp as expected.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

strptime is parsing the timestamp field and expecting it to be in the given format, but you have already said it is a 10 digit number (not the format you are trying to parse with)

0 Karma
Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...