topic Source using ISO Date in Splunk Search

Source using ISO Date

cvajs — Tue, 03 Apr 2012 20:29:20 GMT

v4.3.1 on sles 11.1

i have some syslog-ng data, written to file as template("$DATE $TZ $WEEKDAY $ISODATE $HOST $FACILITY [$LEVEL] $MSG\n")

when i try to create a new source and use ISO Date YYYY-MM-DDTHH:MM:SS.MMM-HH:MM for timestamp format it doesnt find the pattern, keeps locking onto the $DATE format.

why?

Re: Source using ISO Date

sowings — Wed, 04 Apr 2012 02:05:15 GMT

You'll want to look at props.conf, specifically TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD and maybe TIME_FORMAT.

Re: Source using ISO Date

cvajs — Wed, 04 Apr 2012 13:03:34 GMT

i was using the GUI to create a new source definition, etc. when i went to timestamp tab i could not get the GUI to recognize the iso date in the data.

Re: Source using ISO Date

sowings — Thu, 05 Apr 2012 12:36:36 GMT

In the Manager when defining a new sourcetype (in particular, the timestamp tab), TIME_PREFIX corresponds to "Timestamp is always prefaced by a pattern". Since you're writing a "normal" syslog timestamp first, Splunk is finding that. You'll need to specify a regex pattern that matches the "normal" time stamp, so that Splunk will look past this prefix to find the ISO time stamp instead.

TIME_FORMAT is the "Specify timestamp format (strptime)" text entry box. I've successfully used %Y-%m-%dT%H:%M:%S%z.