Log file rotation with date in file name

[1234testtest]

Hi,
I have to get all (and ONLY) tomcat std out files in D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs. I have the following name of tomcat std out file tomcat6-stdout.2013-01-03.log - and it changes everyday. How do I monitor these files continuosly

My inputs.conf has the stanza below.
[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs/tomcat6-stdout*]
disabled = false
followTail = 0
sourcetype = w_std_log_dynamic

I suspect the problem is with log file rotation as if a new tomcat6-stdout with today's date is getting generated, it is not being indexed (not shown in the sources list in the search app).

Thanks and any pointers would be of great help.

Kindly help

[alacercogitatus · Accepted solution]

You could try using a whitelist for the folder. The * in the stanza relates to path, so your monitor was looking for folders instead of files that match the regex.

[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs]
disabled = false
followTail = 0
sourcetype=w_std_log_dynamic
whitelist = tomcat6-stdout.*\.log


[1234testtest]

Thank you. surprisingly after a restart of splunk server - this worked -

[monitor://D:\Program Files\Apache Software Foundation\Tomcat 6.0\logs\tomcat6-stdout*]
disabled = false
followTail = 0
sourcetype = w_std_log_dynamic

[lukejadamec]

This worked for me also. There are many different sourcetypes in this directory, so for this particular sourcetype with the date in the file name is easier to manage as a source with a wildcard than with a whitelist for a directory - I need all files from the directory, but with different sourcetypes.

[alacercogitatus]

The primary reason for not doing it that way is that the assumption was that this was a Windows system "D:" and the documentation states that that will not work on Windows.

Warning: In Windows, you cannot currently use a wildcard at the root level. For example, this does not work:
[monitor://E:\...\foo\*.log]
Splunk logs an error and fails to index the desired files.
This is a known issue, described in the Known Issues topic of the Release Notes. Look there (http://docs.splunk.com/Documentation/Splunk/5.0.1/ReleaseNotes/KnownIssues) for details on all known issues.

[kallu]

How about adding ".log" to the end. Like this

[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs/tomcat6-stdout*.log]

This is then exactly like in examples in Splunk docs (=should work)

http://docs.splunk.com/Documentation/Splunk/latest/Data/Specifyinputpathswithwildcards

Ps. I'm not sure if it makes difference but have you tried turning your "/" to "\" if it is Windows?