Splunk Search

Monitoring a rolling log file

santosh_sshanbh
Path Finder

I have a requirement to monitor a rolling log file from a folder. The name of the file is like below

CalculationMgr-xx(yy).log

Here, xx & yy are the numbers which keeps on changing each time the service restarts. Also for the first time, I do not want to index the old data from the log file but in case the Splunk UF is stopped by any reason, it should not loose the data after it restarts. So can any one help me with the correct Monitor stanza I have to use in this case?

Tags (1)
0 Karma

skoelpin
SplunkTrust
SplunkTrust

Here's a good start

[monitor://<PATH_TO_FILE>/CalculationMgr-*.log]
 index=<YOUR INDEX NAME>
 sourcetype=<YOUR SOURCETYOE>
ignoreolderthan=-1d

You will also need to configure outputs.conf to point to your indexer(s) and restart the splunkd service on the forwarder. The ignoreolderthan attribute will ignore all file older than 1 day, you may want to modify this to fit your use case. Also the fishbucket on the UF will prevent duplication of data

http://docs.splunk.com/Documentation/Forwarder/7.0.3/Forwarder/Configuretheuniversalforwarder
http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/Data/Monitorfilesanddirectorieswithinputs.con...
https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html

0 Karma

santosh_sshanbh
Path Finder

Tried this but getting error in Splunkd

04-04-2018 08:34:03.983 -0400 DEBUG TailingProcessor - Not using stanza for this item (File did not match whitelist '^D:\\Program\ Files\ (x86)\\Proficy\\Proficy\ Server\\LogFiles\\CalculationMgr[^]*.log$'.).

04-04-2018 08:34:03.982 -0400 DEBUG TailReader - Returning disposition=IGNORE_THIS_PATH for file=D:\Program Files (x86)\Proficy\Proficy Server\LogFiles\CalculationMgr-1023(11).Log

UF is Windows 2012 server

0 Karma

santosh_sshanbh
Path Finder

I tried multiple combinations like below, but no success.

[monitor://D:\Program Files (x86)\Proficy\Proficy Server\LogFiles\CalculationMgr*.log]
source = Log
sourcetype = CalculationMgr
recursive = false
followTail = 0
disabled = 0

[monitor://D:\Program Files (x86)\Proficy\Proficy Server\LogFiles]
source = Log
sourcetype = CalculationMgr
recursive = false
whitelist = CalculationMgr-\d+(\d+).log$
followTail = 0
disabled = 0

[monitor://D:\Program Files (x86)\Proficy\Proficy Server\LogFiles]
source = Log
sourcetype = CalculationMgr
recursive = false
whitelist = CalculationMgr-*.log$
followTail = 0
disabled = 0

0 Karma

skoelpin
SplunkTrust
SplunkTrust

It would be helpful if you posted your stanza..

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Classic British Rock Bands

This puzzle series (first published here) is based on extracting information from statements about well-known ...

New to the Community? Join Us for an Exclusive Post .conf26 Welcome Session (Plus a ...

Keep the .conf26 Energy Going: Welcome to the Splunk Community! Whether we scanned your badge at the Community ...

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Register to Attend   Join our Tech Talk ...