I need to add an additional time format for syslog data into the datetime.xml.
Following the docs I have created a copy of datetime.xml and copied it to /system/local,
added the following stanza to it:
I have also created a props.conf in system/local with the following stanza:
DATETIME_CONFIG = /etc/system/local/datetime.xml
Below are example events that I am trying to get the extraction to work on:
-86-2019-01-25T18:26:30.4563+04:00 This is a test event
-86-2019-01-27T19:28:31.4563+00:00 This is a test event
When I try to ingest a file (via the gui) with those events in the file and select syslog as sourcetype I get a warning saying that it could not use strptime to parse the timestamp.
There is a highlight on the even that starts at the beginning of the line but ends at "+04:" , so for some reason it does not include the last two characters (00) in the timezone , even though the regex states that it should include it.
OK, I eventually got to the bottom of this.
The regex that I have put in there is fine , so is the remaining format of the xml file.
I initially discovered that if I create a new sourcetype (and not use "syslog") and get it to use the datetime.xml then it works, so it must have been something in the syslog props definition.
I eventually realised it was the MAX_TIMESTAMP_LOOKAHEAD which is set to 30 for syslog but for any new sourcetype (which will inherit the default settings) it will be set to 128.
The timestamp in the log example that I have ends on the 43rd character of the event.