How can I combine timestamp in the filename plus ms inside the file for an event?

[svn2208]

I read in some old posts, that it is not possible to use the timestamp of a filename, but I wonder, if it is possible meanwhile.

My problem: My filename is something like xxx_20181127_175823.bin.gz my events inside of this log file are based on this "starting point".

Tim=0000023 event xyz
Tim=0000987 event abc

The actual timestamp will be 20181127_175823 + 23 ms or 20181127_175823 + 987 ms, where we have year + month + day + _ + time.

Is it possible to do this with Splunk or do I have to do some workaround?

[mthomas_splunk]

This is possible in Splunk Enterprise 7.2, making use of the new ingest-time eval. Full documentation is at https://docs.splunk.com/Documentation/Splunk/latest/Data/IngestEval.

Example

File Name: xxx_20181127_175823.txt
File Name Format: xxx_%Y%m%d_%H%M%S.txt

props.conf

[mysourcetype]
TRANSFORMS=timestampeval

transforms.conf

[timestampeval]
INGEST_EVAL = _time=strptime(replace(source,"(^.*(?=/)/|\.txt$)","").substr(_raw, 9, 3),"xxx_%Y%m%d_%H%M%S%Q")

This does the following:

1. Takes the the "source" metadata value (which is the path and file name)
2. Removes the path and .txt extension (this leaves the "xxx", date and hours, minutes seconds of the time)
3. Concatenates the 3 characters from the raw event (the milliseconds)
4. Converts the string to a timestamp

All events in the file will have the same day, month, year, hour, minute, second from the file name, but then milliseconds will be from the individual event.