Do I need to use INDEXED_EXTRACTIONS when using FIELD_DELIMITER & FIELD_NAMES on UF?

StephenD1 — Fri, 28 Mar 2025 22:08:27 GMT

I've noticed an issue with one of my syslog indexes. I have a syslog server centralizing and forwarding syslogs for 6 different indexes. Not too long ago, I modified one of the indexes to extract fields at the UF instead of the indexer (this solved another problem that is not relevant here; I can provide detail if it becomes relevant). I noticed that occasionally, that index that is extracting fieldnames at the UF stops sending while the others are sending. The only thing that reliably gets it sending again is restarting the Splunk service on the UF. I'm newish to Splunk so I'm sure I am not troubleshooting all the things I should be. The one thing I noticed is right before the index stops sending, I see errors in the _internal host index

ERROR TailReader [2264207 tailreader0] - Ignoring path="<path/to/log/syslog.log>" due to: Bug during applyPendingMetadata, header processor does not own the indexed extractions confs.

Based on some research here I think I've discovered the problem but I need confirmation before I start making changes. I added the following fields for the sourcetypes:

Forwarder: props.conf:

[sourcetype:here] FIELD_DELIMITER = whitespace FIELD_NAMES = field1,field2,field3,etc...

I think the problem is I did not specify

... INDEXED_EXTRACTIONS = W3C ...

So my question is, do I need the INDEXED_EXTRACTIONS parameter if I use the FIELD_DELIMITER & FIELD_NAMES or can those be used without it?

I believe this is what is missing and causing Splunk to periodically stop processing the file. If I do not need it, then I would need to search for a different cause. Thanks in advance for your help.

Re: Do I need to use INDEXED_EXTRACTIONS when using FIELD_DELIMITER & FIELD_NAMES on UF?

kiran_panchavat — Sat, 29 Mar 2025 02:28:51 GMT

@StephenD1

The FIELD_NAMES and FIELD_DELIMITER attributes only apply when INDEXED_EXTRACTIONS is set.

Please have a look https://community.splunk.com/t5/Getting-Data-In/Error-Bug-during-applyPendingMetadata-header-processor-does-not/m-p/239170

The key here is that you are using INDEXED_EXTRACTIONS. Sourcetypes that use INDEXED_EXTRACTIONS need to have their props.conf on the universal forwarder. There is a good explanation as to the "why" here:

https://community.splunk.com/t5/Getting-Data-In/Why-is-the-sourcetype-specified-in-inputs-conf-on-the-universal/td-p/110285?_gl=1*11h7fp1*_gcl_au*MTY2NzExOTYxOS4xNzQxOTY4Mjky*FPAU*MTY2NzExOTYxOS4xNzQxOTY4Mjky*_ga*MTExNTc3MzM4LjE3NDE5NjgyOTM.*_ga_5EPM2P39FV*MTc0MzIxMzIwOC4xNi4xLjE3NDMyMTUwNjUuMC4wLjE1MTQ4MzIwNzM.*_fplc*U01xUU5CVXpnQmdOems5T1I3YTNFQ3RIODdFRnllRmhWUU5JcXd6bmw5bkh4JTJGS2lKcG9jRm1CbnNSSW80bVBuTmdvdVVhS2RZJTJGeWxCWkJOVk5MMGVMWkhKeDJNQlduOHpXUXFNaHZ4aDMlMkJNOWVhNDRsWGxRbHRGeEMydmNnJTNEJTNE

INDEXED_EXTRACTIONS is a somewhat special processor that is usually done on universal forwarders to ingest structured data. This is done in the parsing queue. The slides for the 2015 conf session are here

https://conf.splunk.com/session/2015/conf2015_ABath_JKerai_Splunk_SplunkClassics_HowSplunkdWorks.pdf

FIELD_DELIMITER = <character>
* Which character delimits or separates fields in the
  specified file or source.
* You can use the delimiters for structured data header extraction with
  this setting.
* This setting supports the use of the special characters described above.
* The default can vary if 'INDEXED_EXTRACTIONS' is set.
* Default (if 'INDEXED_EXTRACTIONS' is not set): not set

FIELD_NAMES = [ <string>,..., <string>]
* Some CSV and structured files might have missing headers.
* This setting tells Splunk software to specify the header field names directly.
* The default can vary if 'INDEXED_EXTRACTIONS' is set.
* Default (if 'INDEXED_EXTRACTIONS' is not set): not set

topic Re: Do I need to use INDEXED_EXTRACTIONS when using FIELD_DELIMITER & FIELD_NAMES on UF? in Getting Data In

Do I need to use INDEXED_EXTRACTIONS when using FIELD_DELIMITER & FIELD_NAMES on UF?

Re: Do I need to use INDEXED_EXTRACTIONS when using FIELD_DELIMITER & FIELD_NAMES on UF?