Splunk File Monitoring

[mohsplunking]

Hello Splunkers,

I have a question around Monitoring a same File from different server, The situation is Server1, Server,2,Server3 is connected to the same NFS where log file abc.log is , now Splunk universal forwarder is installed on all these servers and in the inputs.conf has a monitoring stanza to monitor log file /a/b/c/abc.log, what are the options here to avoid duplication on forwarding/indexing.

Please advise,

Thank !

Moh..

[PickleRick]

If I understand you correctly, you have a file on share exported from an NFS server. This share is mounted on several client machines and contents of the files from that share are being monitored on those machines.

There is no deduplication functionality for ingested data in Splunk (it would be very difficult to do something that would work efficiently and didn't have too many limitations). Especially if the data comes from multiple different sources.

Your best bet would be to make sure you monitor the file only once (possibly from the server itself, not from the client machines).

[isoutamo]

It's exactly like @PickleRick said. Splunk didn't offer any official method to do this kind of deduplication. Then best option is install UF into this nfs server and use it as collecting those. 

Then depending what your actually environment is (there are several possibilities which come into my mind) there could be some other ways to manage it.