I have a logfile with two different date formats for entries. Unfortunately, the dates written to the logfile are "underspecified", and the automatic date extraction is getting them wrong. The following examples are from log.20101010 (log.YYYYMMDD, UTC date), but the individual entries get scattered across several days when they're indexed:
The following ends up on October first:
-- 10/10 01:59:50 [blah, blah, blah]
This one ends up October second:
-- 10/10 02:57:07 [blah, blah, blah]
Another format ended up the correct place today, but just because the 2-digit year happened to match the month and day so it doesn't matter if they get swapped around:al
INFO 10/10/10 02:57:07
The extracted times are all OK.
I've looked at the documentation, and it looks like the precedence rules for extracting dates and times use transforms/extractions from the individual records first, then look for rules to derive the date from the filename.
How can I force the DATE on the indexed record to be derived from the filename? I suspect that datetime.xml may be involved somehow. As an added complication, the logfile is read by a local (not lightweight) agent, passed to a forwarder, and then spread across 4 index servers - so I need to get the right machines as well as the right file(s) to update.
... View more