Getting Data In
Highlighted

How to troubleshoot why my Windows universal forwarders are not recognizing the correct timestamp for my data?

Communicator

Hi,

I have complex events in files forwarded from Windows hosts with Universal Forwarders.
These files are zip-compressed, and have "TRA" in filename.

They look similar to this:

20150422|20150721|grtghtyrt|teghtrhher(... some text)
20150427|20150630|grtghtyrt|teghtrhher(... some text)

Date of each event is THE SECOND column (first is for something else).
So for those 2 events, I expect _time to be 2015-07-21 00:00:00 and 2015-06-30 00:00:00

I made a simple app with props.conf: http://pastebin.com/LGCUNpPp

When I add input directly to Splunk, _time is correct.
When I forward data with Splunk Universal Forwarder, _time is set to modification date of those files, which is wrong.
Sourcetype is set correctly.

Why does Splunk Forwarder ignore my settings? How to debug this and what to do?

0 Karma
Highlighted

Re: How to troubleshoot why my Windows universal forwarders are not recognizing the correct timestamp for my data?

Path Finder

This may be helpful:

The time is extracted where the log data is parsed. This is on the indexer if you are using a lightweight forwarder, and on a forwarder if you are using a heavy forwarder. (Parsing the data is the essential difference between a light and heavy forwarding.)

Update: I wrote this which goes into a more thorough explanation.

Found here: https://answers.splunk.com/answers/612/timestamp-logic-config-on-forwarder-or-indexer.html

View solution in original post

Highlighted

Re: How to troubleshoot why my Windows universal forwarders are not recognizing the correct timestamp for my data?

Communicator

Thank you for the link to wiki. It helped me a lot!

0 Karma
Highlighted

Re: How to troubleshoot why my Windows universal forwarders are not recognizing the correct timestamp for my data?

Communicator

I tried both changes at the same time - and don't know if changing '*' to '...' solved my problem, or moving props to indexers was enough.

Thank you guys!

0 Karma
Highlighted

Re: How to troubleshoot why my Windows universal forwarders are not recognizing the correct timestamp for my data?

Esteemed Legend

According to this:

http://docs.splunk.com/Documentation/Splunk/6.3.1/admin/Propsconf

You need to know this:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met
    or equivalently, matches any number of characters.
*   matches anything but the path separator 0 or more times.
    The path separator is '/' on unix, or '\' on windows.
    Intended to match a partial or complete directory or filename.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So instead of this:

[source::*TRA*]

Try this (or similar) instead:

[source::...TRA...]
Highlighted

Re: How to troubleshoot why my Windows universal forwarders are not recognizing the correct timestamp for my data?

Communicator

Thank you very much

0 Karma