Getting Data In

How to Configure timestamps for events with multiple timestamps

mawwx3
Explorer

I followed the directions for configuring custom timestamps for events with multiple timestamps but I am not getting the result I am looking for. Here is my props.conf in my $Splunk_home$/etc/system/local/ folder:

[host::foo.bar.com]
TIME_PREFIX = \w+ \d+ \d\d:\d\d:\d\d foo.bar.com\s+
TIME_FORMAT = %b %d %H:%M:%S %Y

Here are a couple of entries that I am dealing with:

Jun 14 08:18:20 foo.bar.com Mon Jun 14 08:16:25 2010: 123.123.123.12 -> 231.231.231.23: 43645 NOERR 'a.b.cdf.net.' AAAA IN (x#1)

Jun 14 08:18:20 foo.bar.com Mon Jun 14 08:16:25 2010: 124.124.124.12 -> 232.232.2.232: 14267 NOERR 'b.somestuff.net.' A IN (a#1) (n#4) (x#4) ANS abc.somestuff.net. A IN 213.12.213.123

I would like the timestamp to correspond to the time given after foo.bar.com but the timestamp is shown as the time at the beginning of each entry before foo.bar.com.

Any help would be appreciated.

Tags (2)
1 Solution

hulahoop
Splunk Employee
Splunk Employee

Hi Michael,

Are you setting the host value in another props.conf stanza? If so, then your timestamping rules do not get honored. At index-time, Splunk makes only one pass through props.conf. If during the first pass, your host (foo.bar.com) does not yet exist, then the timestamping rules are ignored.

Your timestamp rules look to be correct and works when I tested it on the 2 sample events. The only difference is I set the rules using the sourcetype, not the host. Is it possible to use [sourcetype] instead of [host::foo.bar.com]?

View solution in original post

hulahoop
Splunk Employee
Splunk Employee

Hi Michael,

Are you setting the host value in another props.conf stanza? If so, then your timestamping rules do not get honored. At index-time, Splunk makes only one pass through props.conf. If during the first pass, your host (foo.bar.com) does not yet exist, then the timestamping rules are ignored.

Your timestamp rules look to be correct and works when I tested it on the 2 sample events. The only difference is I set the rules using the sourcetype, not the host. Is it possible to use [sourcetype] instead of [host::foo.bar.com]?

hulahoop
Splunk Employee
Splunk Employee

[manual] should work fine. Technically these events are not formatted in the standard syslog format.

0 Karma

mawwx3
Explorer

I have my sourcetype set to manual for the port I have listening for this data. Can I just use [manual] then in props.conf or should I change the sourcetype?

0 Karma

hulahoop
Splunk Employee
Splunk Employee

In that case, then try [syslog] instead of [host::foo.bar.com] in props.conf and restart Splunk. Keep in mind, the timestamping rules will only apply to new incoming events, and will not 'fix' timestamps retroactively for events which have already been indexed.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

it is very likely that the host that you see in the event (foo.bar.com) is being set because your sourcetype is syslog. the actual host for a syslog event may or may not be the same.

0 Karma

mawwx3
Explorer

The only other stanzas I have in my props.conf file are eventtype stanzas that relate to creating custom fields with the same host. I have stanzas in eventtypes.conf and transforms.conf accordingly for the eventtype stanzas. I am still trying to get the props.conf file down, so how do I use[sourcetype] in the props.conf file as you say?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...