Getting Data In

Timestamp and Universal Forwarder confusion

PickleRick
SplunkTrust
SplunkTrust

I'm a bit lost. Every piece of info that I find on the web (as well as materials from the Splunk's own trainings) say that UF does only very limited input preparation (line breaking, metadata adjustment, character encoding) but no real parsing work.

Thus I'm confused to find in logs that, for example:

12-03-2021 15:50:44.906 +0100 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Dec 3 15:50:36 2021). Context:[...]

That would mean that some timestamp parsing does take place.

But do I still need to put my timestamp extraction config on the HF? (I use UF -> HF-> idx)

How does it correspond to possible settings about breaking on timestamp? If I want to break (which should happen on UF, right?) on timestamp, do I need to provide timestamp format on both UF (for breaking) and HF (for parsing)?

Labels (2)
0 Karma

harsmarvania57
Ultra Champion

It depends, if you have sourcetype which is using INDEXED_EXTRACTIONS then those data goes through StructuredParsing pipeline which means datetime parsing happen on UF. See https://wiki.splunk.com/Community:HowIndexingWorks (4. Detail Diagram - UF/LWF to Indexer)

PickleRick
SplunkTrust
SplunkTrust

This UF is ingesting Exchange logs. If I remember correctly, they are structured but they don't use indexed extractions (but I might be wrong; I don't have this evironment at hand to check it quickly).

But in general, if I want to add another sourcetype, with no indexed extractions, I should normally add timestamp-related props on HF, right?

Thanks for the link!

0 Karma
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...