TIME_FORMAT UDP Input Fail

[matthewparry]

Hi,

I am trying to use a different timestamp located in the event data. This is a UDP input on a forwarder which I have first created a new sourcetype.

props

[source::udp:514]
TRANSFORMS-changesource = new_sourcetype

transforms

[new_sourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = mydata
FORMAT = sourcetype::newsourcetype

Then on my indexer splunk is automatically appending the date + host to my events coming from UDP, this can be stopped by using "no_appending_timestamp = true" under the input and this actually causes splunk to see the correct date from the raw data.

I believe you cannot apply timestamp changes to sourcetypes that have been generated in a transform?

I have tried the following on the indexer but with no luck:

props

[host::192.168.1.1]
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT = %b %d %H:%M:%S.%6N

Example data coming in via syslog-ng

Jul 30 12:57:02.68787 host1 testdaemon: DEBUG Starting daemon

This gets indexed

Jul 30 12:57:02 192.168.1.1 Jul 30 12:57:02.687871 host1 testdaemon: DEBUG Starting daemon

Help would be appreciated as I have rattled my brain for hours now! 🙂

[jbsplunk · Accepted solution]

On your heavy forwarder, you've made some mistakes.

For props.conf, I would add TIME_PREFIX with a regex to match the data that comes before the timestamp you want to extract

[host::192.168.1.1]
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_PREFIX = REGEX
TIME_FORMAT = %b %d %H:%M:%S.%6N

This transform is incorrect:

FORMAT = sourcetype::newsourcetype

What you should be doing looks like this:

[new_sourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = mydata
FORMAT = newsourcetype

[matthewparry]

I am actually using:

[host::192.168.1.1]
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_PREFIX = .+(?=\w{3} \d{2} \d{2}:\d{2}:\d{2}.\d{6})
TIME_FORMAT = %b %d %H:%M:%S.%6N

This has worked. I am unable to get it too work by substituting the [host::192.168.1.1] for a sourcetype but I believe this is due to the fact I am using a transform to generate a new sourcetype and the timestamps need to come before any transforms..

[jbsplunk]

only for search time field extractions, which I don't think is what is intended here:

FORMAT for search-time extractions:
* The format of this field as used during search time extractions is as follows:
* FORMAT = ::( ::)*
* where:
* field-name = [|$]
* field-value = [|$]
* Search-time extraction examples:
* 1. FORMAT = first::$1 second::$2 third::other-value
* 2. FORMAT = $1::$2

[sowings]

This document suggests that the "sourcetype::" is in fact needed.

[matthewparry]

I am trying to use the time from syslog-ng "Jul 30 12:57:02.687871" as the event time.

[Drainy]

No. once the sourcetype has been changed then it will have passed the index time parsing.
What you want to do is apply the timestamp stripping from the syslog message first and then change the sourcetype, e.g;

[source::udp:514]
TRANSFORMS-stripts = strip_timestamp
TRANSFORMS-changesource = new_sourcetype

You could still do it on a per host basis though, there may be another way to wriggle it round but this is the way I've ended up doing it on my implementations

[matthewparry]

I have a heavy forwarder that receives the UDP syslog data which then forwards this to the indexer.
For some reason when trying to create the source type, this needs to be on the forwarder? and the timestamp changes need to be done on the indexer?

This means it will never execute in the correct order?

[sowings]

You're trying to use the syslog-ng stamp at the beginning of the line, or the device's native stamp starting after its IP address?