Getting Data In

universal forwarder not forwarding all the files to indexer

lakshman237
Path Finder

I had the stanza in inputs.conf in the universal forwarder as:
[monitor:///my/logs/project]
blacklist = .(gz)$
whitelist = (xyz_debug_ms[1-4]{1}.txt|app1_ms[1-4]{1}.txt|\
app2sos_ms[1-2]{1}.log|system_ms.log\
remoteserver.log)
sourcetype = mylogs
index = my index

After restart, the forwarder showed only a few files in "splunk list monitor" and only those files were sent to indexer for search. I then removed "\" and create two stanza, with same monitor:: line, with a few files in whitelist in the first stanza and the remaining in the second stanza.

After restart, the forwarder is not showing the files which it had shown earlier in the list monitor. how to ensure all the files can be monitored and send to indexer?

Tags (1)
0 Karma

itinney
Path Finder

Your blacklist should probably be:

blacklist = \.gz$

I believe your whitelist is missing a pipe symbol between the last two file specs, it should probably be:

whitelist = (xyz_debug_ms[1-4]{1}.txt|app1_ms[1-4]{1}.txt|app2sos_ms[1-2]{1}.log|system_ms.log|remoteserver.log)

When you say you then created two stanza, can you include them so we can see what they look like? You should not have overlapping monitor stanzas.

lakshman237
Path Finder

The pipe was missed when I posted the query, however it's present in my config spec. Any other possibilities?

Also, does splunk not allow to 2 or more stanza with the same monitor line, eg. monitor::/var/log monitor::/var/log with different files in the whitelist?

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...