topic Re: How to set a different time format? in Getting Data In

How to set a different time format?

pfabrizi — Wed, 07 Mar 2018 15:25:15 GMT

I am trying to format the time that is in this format: [dd/mmyyyy HH:MM:SS GMT] when I set the time_prefi to a regex that contains [ it seems to stop the data from being indexed. When remove the settings it start collecting.

I am guessing this is because a [ is regex command? Is another way to do this?

Thanks!

Re: How to set a different time format?

somesoni2 — Wed, 07 Mar 2018 15:47:40 GMT

Can you provide a full data sample and your current props.conf configuration for your sourcetype?

Re: How to set a different time format?

skoelpin — Tue, 29 Sep 2020 18:15:29 GMT

Time_prefix is an attribute of base configs which should be applied to every sourcetype on the indexers.

Time_prefix works by identifying where the timestamp is located in your logs so the TIME_FORMAT attribute can see what format the timestamp is in.

There's 3 attributes that help get the timestamp correct for your specified sourcetype. An example would look like this

03/01/2018 13:05:59 - INFO dfgdsfgdgerrfr

[sourcetype]
TIME_PREFIX = ^ 
TIME_FORMAT = %m/%d/%Y\s%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20

Time_prefix shows where the sourcetype is in the logs
TIME_format shows the format of the timestamp
MAX_timestamp_lookahead identifies how far in your log Splunk should look to identify the timestamp.. Any props.conf changes on the indexer will require a splunkd restart

Re: How to set a different time format?

pfabrizi — Tue, 29 Sep 2020 18:20:00 GMT

so this is the event:
Mar 7 11:43:30 xxxxxcxxxx.xxxx.net [07/03/2018: 16:43:29 GMT]

stanza:
TIME_PREFIX = net\s[
TIME_FORMAT %d/%m/%Y: %H:%m:%S
MAX_TIMESTAMP_LOOKAHEAD = 20

this was stopping index of these events. when I removed these from my stanza it would start again. I was not sure if '[' was causing an issue,

Thanks!

Re: How to set a different time format?

skoelpin — Wed, 07 Mar 2018 16:51:59 GMT

Your TIME_FORMAT is wrong. Its missing an equal sign and the minute part is wrong

It should look like this

TIME_FORMAT = %d/%m/%Y: %H:%M:%S

Your TIME_PREFIX is also wrong and should look like this

TIME_PREFIX = net\s\[

You should not remove this as your putting more overhead on the indexer

Re: How to set a different time format?

FrankVl — Wed, 07 Mar 2018 17:12:55 GMT

[ plays a special role in regular expressions, as the start of a character set notation (e.g. [a-z]). So if you want to match the literal [ character, you need to escape it:
TIME_PREFIX = net\s\[

That, plus the comments from @skoelpin on the time format part should get your config working.

Re: How to set a different time format?

pfabrizi — Wed, 07 Mar 2018 17:56:39 GMT

i had to have been missing something in my original attempt. my second attempt was what you have in there and it wasn't working. I fo have issue from time to time where a change requires me to restart splunk and not just deploy-server.

Re: How to set a different time format?

skoelpin — Wed, 07 Mar 2018 17:58:09 GMT

Your missing a lot more than just the TIME_PREFIX..

Before you continue using Splunk, you should really look into the Splunk education courses as this is very basic stuff