Getting Data In

Epoch Time

hmrabet2
Observer

Im onboarding sample logs from a txt file to my local Splunk instance were the time stamp is in a 10 digit format (epoch time format). During the onboarding im applying the following timestamp format  strptime("timestamp","%m/%d/%y %H:%M:%S") "timestamp" being the field name in the raw sample in the txt document.  But the timestamp is still defaulting to modtime. Any ideas? 

 

Labels (1)
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

TIME_FORMAT=%s is the proper way to configure a timestamp in epoch format. If your logs are formatted such that Splunk cannot clearly identify which 10-digit value represents a timestamp, you may need to provide more hints (recommended to be explicit anyways), like TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD etc.

If you are able to provide a sample log event, it will be easier to help with more details.

0 Karma

hmrabet2
Observer

Example timestamp in raw logs:        timestamp1617865161

0 Karma

s2_splunk
Splunk Employee
Splunk Employee
TIME_PREFIX=timestamp:\s
TIME_FORMAT=%s

should extract the timestamp properly

0 Karma

hmrabet2
Observer

Thanks, i have added the below to the advanced  section under timestamp but its still defaulting back to modtime. 

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Please share one full event here for further help. You can anonymize data as needed, but please maintain the format of the event.

0 Karma

hmrabet2
Observer

Anonymised raw sample: 

{"hostname":"ip-xxx-xxx-xxx-xx.eu-west-1.compute.internal","query":"xxxxx.net.","response_code":"NXDOMAIN","size":"89","src_ip":"xx.xxx.xxx.xxx","timestamp":"1617865214"}

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

OK, it's JSON format, that's helpful to know....

[yourSourcetypeName]
SHOULD_LINEMERGE=false
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT = %s

 You may have to add other settings here depending on other requirements, like line breakers etc., but this should parse your epoch timestamp as expected.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

strptime is parsing the timestamp field and expecting it to be in the given format, but you have already said it is a 10 digit number (not the format you are trying to parse with)

0 Karma
Get Updates on the Splunk Community!

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...

Stay Connected: Your Guide to October Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...