Getting Data In

universal forwarder not forwarding all the files to indexer

lakshman237
Path Finder

I had the stanza in inputs.conf in the universal forwarder as:
[monitor:///my/logs/project]
blacklist = .(gz)$
whitelist = (xyz_debug_ms[1-4]{1}.txt|app1_ms[1-4]{1}.txt|\
app2sos_ms[1-2]{1}.log|system_ms.log\
remoteserver.log)
sourcetype = mylogs
index = my index

After restart, the forwarder showed only a few files in "splunk list monitor" and only those files were sent to indexer for search. I then removed "\" and create two stanza, with same monitor:: line, with a few files in whitelist in the first stanza and the remaining in the second stanza.

After restart, the forwarder is not showing the files which it had shown earlier in the list monitor. how to ensure all the files can be monitored and send to indexer?

Tags (1)
0 Karma

itinney
Path Finder

Your blacklist should probably be:

blacklist = \.gz$

I believe your whitelist is missing a pipe symbol between the last two file specs, it should probably be:

whitelist = (xyz_debug_ms[1-4]{1}.txt|app1_ms[1-4]{1}.txt|app2sos_ms[1-2]{1}.log|system_ms.log|remoteserver.log)

When you say you then created two stanza, can you include them so we can see what they look like? You should not have overlapping monitor stanzas.

lakshman237
Path Finder

The pipe was missed when I posted the query, however it's present in my config spec. Any other possibilities?

Also, does splunk not allow to 2 or more stanza with the same monitor line, eg. monitor::/var/log monitor::/var/log with different files in the whitelist?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...