This is not an "answer" but more of a list of things I've tried on my system with no workable solutions, so I'm calling on the help of those more knowledgeable gurus, like Gerald.
I confirmed that both the following timestamp format strings to do not work, just as Jeffa has reported:
TIME_FORMAT = %b %d, %Y\t%H:%M:%S:%3N
TIME_FORMAT = %b %d, %Y\t%H:%M:%S:%Q
Knowing that the sub-seconds is actually stored in the indexed _subsecond field, I decided to try a more exotic workaround by populating the field explicitly:
So I used a TIME_FORMAT with no sub-seconds, then used a transformer to modify the _raw event to add in one or two leading 0 s (so it is properly padded to 3 digits), and then used a final transformer to extract out the value of _subsecond , however doing this seems to result in splunk completely dropping the event all together. I think this may be because splunk is still extracting a subseconds value even when no %3N or %Q is actually used in the TIME_FORMAT string. So therefore, I suspect that my attempt to set _subsecond manually is infact leading to two _subsecond values which I'm guessing is what splunk doesn't like and is therefore dropping it (This is just a theory)
Here is the config I tried:
props.conf:
[source::.../timestamp_test.log]
sourcetype = timestamp_test
[timestamp_test]
TIME_FORMAT = %b %d, %Y\t%H:%M:%S
SHOULD_LINEMERGE = False
TRANSFORMS-subseconds = fixup_subsecond, fixup_subsecond, extract_subsecond
transforms.conf:
[fixup_subsecond]
REGEX = ^(\S+\s+\w{3} \d+,\s+\d{4}\s+\d\d:\d\d:\d\d):(\d\d?\s.*)$
FORMAT = $1:0$2
DEST_KEY = _raw
[extract_subsecond]
REGEX = ^\S+\s+\w{3} \d+,\s+\d{4}\s+\d\d:\d\d:\d\d:(\d\d\d)\s
FORMAT = _subsecond::$
WRITE_META = True
This config actually prevents the events from being indexed at all. But if you change, the format line to FORMAT = the_sub_second::$1 (for example) then the event does get indexed but obviously it still doesn't have the correct sub-second timestamp.
On a further test, I determined that if you modify the example data and replace the " : " in the timestamp between seconds and subseconds with a space (" "), and then adjust the all the regexes and formats appropriately, then the above extract-your-own-subseconds-hack acutally does work. Of course, this still doesn't work, because if reformatting the input were an option, the obvious thing to do would be to pad the sub-seconds properly. So that's it, I'm out of ideas.
... View more
