Getting Data In

Help with timestamp and time_prefix

a212830
Champion

Hi,

I'm having some issues getting a feeds timestamp picked up properly. The date field comes in like this: "date": "8/15/2019 10:55:16 AM". My props has this, which isn't working.

TZ = UTC
TIME_FORMAT = %m/%d/%Y %I:%M:%S %p
TIME_PREFIX = "date": "
0 Karma

sloshburch
Ultra Champion

Give a btool output of the props.conf so we can see what's really happening. I'm guessing the above is from the props.conf without btool.

0 Karma

woodcock
Esteemed Legend

You need this for props.conf:

TZ = UTC
TIME_PREFIX = "date":\s+"
TIME_FORMAT = %m/%d/%Y %I:%M:%S %p

Your settings are fine so it must be something else. If you are doing a sourcetype override/overwrite, you must use the ORIGINAL value, NOT the new value. You must deploy your settings to the first full instance(s) of Splunk that handle the events (usually either the HF tier if you use one, or else your Indexer tier), restart all Splunk instances there, send in new events (old events will stay broken), then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly indexed events.

0 Karma

Sukisen1981
Champion

date i coming as per the format specified, how would you like to see the date?

0 Karma

a212830
Champion

It's not taking that date as the event date - looks like it's taking the system date on the server.

0 Karma

Sukisen1981
Champion

have you set the spec?
[]
DATETIME_CONFIG =
TIME_PREFIX =
MAX_TIMESTAMP_LOOKAHEAD =
TIME_FORMAT =
TZ =
MAX_DAYS_AGO =
MAX_DAYS_HENCE =
MAX_DIFF_SECS_AGO =
MAX_DIFF_SECS_HENCE =

In this syntax, can be:

, the source type of an event.
host::, where is the host value for an event.
source::, where is the source value for an event.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Configuretimestamprecognition

0 Karma

a212830
Champion

I have what's listed above, which works most of the time. Trying to determine if something there is wrong, especially since the time prefix includes quotes.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!