Getting Data In

How can I index events with the real date while using the timestamp from the log?

ZimmermanC1
Explorer

I have a Splunk instance in a Development & Test lab that uses what we call "repeatable time" to test software updates against a known good checkpoint of data. During a test all of the server times are rolled back to March 16 2011. I have been struggling to figure out how to get Splunk to ignore the Date in the logged events and only care about the "time".

The logs are being monitored in a directory structure as follows /var/adm/splunk/2017_"today's_julian_date"/"server_name"/*.log

 Example Log: header,123,321,2011-03-16 17:35:36.035 +00:00,subject,.............

Effectively I am trying to get these events indexed with today's date but the Time from the log. so it would have:

  _time=2017-12-11 17:35:36.035

I have tried playing with the TIME_PREFIX & MAX_TIMESTAMP_LOOKAHEAD settings in the sourcetype but have not been successful.

 TIME_PREFIX = ^[^\s]+\s
 MAX_TIMESTAMP_LOOKAHEAD = 25

This is a critical issue for me to sort out so any help would be greatly appreciated.

0 Karma

ZimmermanC1
Explorer

Following some more research I was able to determine that the date portion of the timestamp was being generated based on the file name which was called ASCII.20110316....... So i created a script to rename each file to ACII.2017-12-13....... and it appears to now almost be working. The problem seems to be that Splunk is now timeshifting the logs one day into the future. I assume it has something to do with the timezone but I am not 100% sure.

Any other ideas would be greatly appreciated.

0 Karma

DalJeanis
Legend

You might want to check out the Splunk Event Gen app, which IIRC can take input data and modify the timestamp in the _raw data for you.

https://splunkbase.splunk.com/app/1924/

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...