Tomcat log rotation question

[Branden]

I read a similar post about this here but I'm not sure I completely understand.

My tomcat log name looks like this: tomcat_access_log.2010-09-08.txt

I understand I can just monitor for /tomcat/logs/tomcat_access_log* (right?)

When it gets indexed, I don't want the source to be called "tomcat_access_log.2010-09-08.txt". I'll end up with hundreds more sources. I'd rather it simply be called "tomcat_access_log" or something like that.

So let me take a stab at this:

In inputs.conf:

[monitor:///tomcat/logs/tomcat_access_log*]
sourcetype = tomcat-access

In props.conf:

[tomcat-access]
TRANSFORMS-rename_source = drop_trailing_text

In transforms.conf:

[drop_trailing_text]
DEST_KEY   = MetaData:Source
SOURCE_KEY = MetaData:Source
REGEX    = source::(.*)\d\d\d\d-\d\d-\d\d.txt
FORMAT   = source::$1

I guess it's the REGEX I'm uncertain about.

Am I on the right track?

Thanks!

[Lowell · Accepted solution]

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

[Branden]

Worked GREAT, thanks!