Getting Data In

Epoch Time

hmrabet2
Observer

Im onboarding sample logs from a txt file to my local Splunk instance were the time stamp is in a 10 digit format (epoch time format). During the onboarding im applying the following timestamp format  strptime("timestamp","%m/%d/%y %H:%M:%S") "timestamp" being the field name in the raw sample in the txt document.  But the timestamp is still defaulting to modtime. Any ideas? 

 

Labels (1)
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

TIME_FORMAT=%s is the proper way to configure a timestamp in epoch format. If your logs are formatted such that Splunk cannot clearly identify which 10-digit value represents a timestamp, you may need to provide more hints (recommended to be explicit anyways), like TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD etc.

If you are able to provide a sample log event, it will be easier to help with more details.

0 Karma

hmrabet2
Observer

Example timestamp in raw logs:        timestamp1617865161

0 Karma

s2_splunk
Splunk Employee
Splunk Employee
TIME_PREFIX=timestamp:\s
TIME_FORMAT=%s

should extract the timestamp properly

0 Karma

hmrabet2
Observer

Thanks, i have added the below to the advanced  section under timestamp but its still defaulting back to modtime. 

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Please share one full event here for further help. You can anonymize data as needed, but please maintain the format of the event.

0 Karma

hmrabet2
Observer

Anonymised raw sample: 

{"hostname":"ip-xxx-xxx-xxx-xx.eu-west-1.compute.internal","query":"xxxxx.net.","response_code":"NXDOMAIN","size":"89","src_ip":"xx.xxx.xxx.xxx","timestamp":"1617865214"}

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

OK, it's JSON format, that's helpful to know....

[yourSourcetypeName]
SHOULD_LINEMERGE=false
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT = %s

 You may have to add other settings here depending on other requirements, like line breakers etc., but this should parse your epoch timestamp as expected.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

strptime is parsing the timestamp field and expecting it to be in the given format, but you have already said it is a 10 digit number (not the format you are trying to parse with)

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...