Getting Data In

Deployment App inputs.conf issues

Norling80
Path Finder

Hi.

We have an issue when we are trying to collect syslogdata from a filesystem on a syslog server with help from a deployment-app.

The folder structure looks like this, all data is being written to /var/log/netlogs/"IP-adress of the device"/messages.log

Example structure:
/var/log/netlogs/10.3.56.12/messages.log
/var/log/netlogs/10.3.56.13/messages.log
/var/log/netlogs/10.3.56.14/messages.log
/var/log/netlogs/10.3.56.15/messages.log

Here is the inputs.conf file we are using right now (we have experimented back and forward with crcSalt, initCrcLength)
[monitor:///var/log/netlogs/]
sourcetype = syslog
whitelist = .log$
SHOULD_LINEMERGE = false
crcSalt = <string>
initCrcLength = 6250
ignoreOlderThan = 10m
index = network

Any ides what we need to do in order to index every single messages.conf in this filestructure

Tags (1)
0 Karma

samhughe
Path Finder

Have you looked at https://wiki.splunk.com/Community:Troubleshooting_Monitor_Inputs in particular https://your-splunk-server:8089/services/admin/inputstatus/TailingProcessor:FileStatus ? Can be quite helpful in seeing why files aren't being processed.

My guess would be "ignoreOlderThan = 10m" is the issue. This means that Splunk ignores any files that is older than 10minutes when it first sees the file (on startup) even if it is written to later on while Splunk is still running. The only way to get Splunk to process the files once it has 'tagged' them as old, is to restart Splunk.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

How about just this

[monitor:///var/log/netlogs/*/messages.log]
sourcetype = syslog
SHOULD_LINEMERGE = false
crcSalt = <string>
initCrcLength = 6250
ignoreOlderThan = 10m
index = network
0 Karma

Norling80
Path Finder

Thanks for the answer, unfortunately that did not do the trick, we have also checked the filesystem access rights and they are all good.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...