topic Re: Timestamp extractor issue on UF in Getting Data In

Timestamp extractor issue on UF

LIS — Thu, 21 Aug 2025 11:36:13 GMT

I have two time stamps in each record 2025-08-20 17:37:00.317 and SEN_20250820153640.1703351.txt.

And want to use first one, but timestamp extractor chooses second from file name, neglecting my config.

Example of record:

2025-08-20 17:37:00.317 INF transferred file bytes=7642 file=/map/sen-tig/SEN_20250820153640.1703351.txt pid=4170 speed_mbps=2.1 time_ms=29.2

inputs.conf

[monitor:///apps/sftp_sender/var/log/sftp*.log]
disabled = false
index = messaging
sourcetype = messaging:ms:log

props.conf

[messaging:ms:log]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 23
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TZ = Europe/Brussels

I also tried to use DATETIME_CONFIG = NONE (and CURRENT) , but it doesnt work.

I still see in Splunk _time = "second timestamp from record".

props.conf

[messaging:ms:log]
TZ = Europe/Brussels
DATETIME_CONFIG = NONE

Re: Timestamp extractor issue on UF

gcusello — Thu, 21 Aug 2025 15:56:55 GMT

Hi @LIS ,

where did you locate the props.conf file?

you must put it on the UF and on the first full Splunk instance.

Ciao.

Giuseppe

Re: Timestamp extractor issue on UF

PickleRick — Thu, 21 Aug 2025 17:20:18 GMT

https://community.splunk.com/t5/Getting-Data-In/Diagrams-of-how-indexing-works-in-the-Splunk-platform-the-Masa/m-p/590774

Timestamp recognition happens usually on the first "heavy" component the event goes through. So your time-related settings should be put there (usually on indexer(s) or a Heavy Forwarder if you have one between your UF and indexer(s)). There is one caveat though - if you use indexed extractions - they happen on the initial ingesting component even if it's a UF. But that's a rare use case.

Re: Timestamp extractor issue on UF

LIS — Thu, 21 Aug 2025 18:26:43 GMT

Hi @PickleRick and @gcusello ,

Thank you for your responses, but it is not clear for me, because in our system we have lots of configs/dataflows were we have props.conf with time related parameters (TIMESTAMP_FIELDS, TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD, TIME_FORMAT, TZ .... ) only on UF. Our forwarders connected to indexers directly, we dont have any props.conf on indexers and it work properly).

maybe it only works with INDEXED_EXTRACTIONS ?

INDEXED_EXTRACTIONS = csv
FIELD_DELIMITER = ,
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
disabled = false
TIMESTAMP_FIELDS = date
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TZ = UTC

Re: Timestamp extractor issue on UF

PickleRick — Thu, 21 Aug 2025 20:54:02 GMT

If you don't have the proper config on the parsing component Splunk guesses. Sometimes does it relatively well but it's not good performancewise.

Also TIMESTAMP_FIELDS makes sense only with indexed extractions.

Re: Timestamp extractor issue on UF

gcusello — Thu, 21 Aug 2025 21:34:09 GMT

Hi @LIS ,

some pasring activities are done on the Forwarders and some others are done on the first full Splunk instance that data are passing through, in your case Indexers.

Put the props.conf on the UFs and on Indexersa, and, if you have, on Search Heads.

Ciao.

Giuseppe

Re: Timestamp extractor issue on UF

LIS — Fri, 22 Aug 2025 07:50:28 GMT

Thank you for more details).

Unfortunately, it is not really clear in Splunk docs where we can use certain parameter (only UF, HF, Indexer or on any of them). how to get it?

So, I have to copy my props.conf on indexers as well, or move it from UF to indexers?

Thank you in advance:-)

Re: Timestamp extractor issue on UF

gcusello — Fri, 22 Aug 2025 08:04:07 GMT

Hi @LIS ,

I usually put it both on UFs and Indexers, and Search Heads.

Ciao.

Giuseppe

Re: Timestamp extractor issue on UF

LIS — Fri, 22 Aug 2025 10:02:50 GMT

Hi @gcusello and @PickleRick ,

I copied props.conf to indexers and it works properly :-).

Thank you and have a nice day!

Re: Timestamp extractor issue on UF

gcusello — Fri, 22 Aug 2025 10:07:13 GMT

Hi @LIS ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉