how to extact multiple timestamp formats for syslog input

[alextsui]

Hi, I have setup Splunk to listen on udp:514 for syslog input and run into a problem when some logs have single timestamp information and others have multiple timestamp information within the logs.

sample log 1 : Jan 31 14:45:17 10.10.10.11 postfix/cleanup[32495]:.........(omitted)

sample log 2 : Jan 31 14:46:12 10.10.10.10 Jan 31 14:50:50 Forwarded........(omitted)

I need to extract the second timestamp in sample log 2 (10.10.10.10 is extracted to be the host by Splunk) and have tried using the following configuration without success.

inputs.conf-

[udp://514]
disabled = false
connection_host = ip
sourcetype = syslog

props.conf -

[host::10.10.10.10]
TIME_PREFIX = \w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+
TIME_FORMAT = %b %d %T
MAX_TIMESTAMP_LOOKAHEAD = 50

From another Splunk Answers post "How to Configure timestamps for events with multiple timestamps" gkanapathy mentioned "it is very likely that the host that you see in the event (foo.bar.com) is being set because your sourcetype is syslog. the actual host for a syslog event may or may not be the same". If this were true how do I apply the timestamp extraction config by host? OR is there a workaround?

Thanks.

[gkanapathy]

If there are multiple timestamps, you can use a custom DATETIME_CONFIG instead of specifying TIME_FORMAT and TIME_PREFIX. This is not heavily documented, but basically, you create a custom version of the $SPLUNK_HOME/etc/datetime.xml file (remove the default patterns, and insert the specific patterns that you want to match), then set DATETIME_CONFIG to point to this new custom file instead of using TIME_FORMAT/TIME_PREFIX.

Note that if TIME_FORMAT fails to match or is not specified, Splunk will fall through and try the DATETIME_CONFIG file to find a timestamp in an event. The default Splunk datetime.xml has several common patterns, and is what is used by Splunk to "guess" at timestamps in an event. If you have a specific set of patterns, you can make the timestamp extraction more precise, more controlled, and less CPU-intensive (and faster) with a custom DATETIME_CONFIG.

[gkanapathy]

You can apply it to the hosts, provided you know the host name ahead of time, and it's the host name that Splunk sees coming in before any TRANSFORMS are applied. Note that syslog sourcetypes usually TRANSFORM the host to whatever is in the event text, so what is indexed is not what Splunk sees coming in.

[alextsui]

Can I apply the DATETIME_CONFIG to hosts in the props.conf? For exampel:
[host::10.10.10.10]
DATETIME_CONFIG = /datetime.xml

I thought the problem was not able to apply to host becuase the host name may or may not be the ones I see on the search apps' main dashboard?