topic Re: How do I edit my props.conf for proper timestamp extraction from my sample log entry? in Getting Data In

How do I edit my props.conf for proper timestamp extraction from my sample log entry?

mark19632 — Mon, 14 Dec 2015 23:13:25 GMT

I'm having trouble with a log and getting Splunk to recognize the time format.

Here is an example a log entry:

010406:00:530000000000000040RD000001071215

Now, all the entries start with 0104 followed by the time in H:M:S format.

I've added a props.conf to the indexer like this:

[sisfeedlog]
TIME_PREFIX = ^0104
TIME_FORMAT = %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 8

and a props.conf on the server:

[source::...\\SISFeed\\S(\d+\.LOG)]
sourcetype = sisfeedlog

It doesn't seem to be working though as the time isn't being extracted and the sourcetype is coming up as unknown.

Any advise on why it's not working?

Thanks,

Mark

Re: How do I edit my props.conf for proper timestamp extraction from my sample log entry?

MuS — Tue, 15 Dec 2015 00:14:08 GMT

Hi mark19632,

when defining a stanza in props.conf you can only use a limited set of regexes:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met
    or equivalently, matches any number of characters.
*   matches anything but the path separator 0 or more times.
    The path separator is '/' on unix, or '\' on windows.
    Intended to match a partial or complete directory or filename.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So, your [source::...\\SISFeed\\S(\d+\.LOG)] will not work since \d+ is not supported.
What would work, is this [source::...\\SISFeed\\S*.LOG] but I don't know if this is matching the correct directory..... BTW why don't you set the sourcetype in the servers inputs.conf?

Hope this helps ...

cheers, MuS

PS: your props.conf on the indexer looks fine.

Re: How do I edit my props.conf for proper timestamp extraction from my sample log entry?

mark19632 — Tue, 29 Sep 2020 08:07:30 GMT

Thanks MUS, I've added it to the inputs.conf and picking up the sourcetype and the time correctly 🙂

Now for the next problem!

The log files are named s*date*.log eg S15122015 .log for today.

When read in the logs are showing as the incorrect date, so 06122015 is being picked up as 12th of July rather than the 6th of December.

Can I change this?

Thanks,

Mark

Re: How do I edit my props.conf for proper timestamp extraction from my sample log entry?

MuS — Tue, 15 Dec 2015 19:49:29 GMT

use the TIME_FORMAT = option in props.conf to set the format of your time string to something like this

 TIME_FORMAT = %d%m%Y

see the docs for more details about the time strings http://docs.splunk.com/Documentation/Splunk/6.3.1/SearchReference/Commontimeformatvariables and the props.conf setting http://docs.splunk.com/Documentation/Splunk/6.3.1/Admin/Propsconf

Re: How do I edit my props.conf for proper timestamp extraction from my sample log entry?

mark19632 — Wed, 16 Dec 2015 09:00:39 GMT

I've tried that and it's made no difference unfortuantly

Re: How do I edit my props.conf for proper timestamp extraction from my sample log entry?

mark19632 — Thu, 17 Dec 2015 14:05:58 GMT

I fixed it by creating a custom datetime.xml in the end 🙂