Archive

Tomcat log rotation question

Builder

I read a similar post about this here but I'm not sure I completely understand.

My tomcat log name looks like this: tomcat_access_log.2010-09-08.txt

I understand I can just monitor for /tomcat/logs/tomcat_access_log* (right?)

When it gets indexed, I don't want the source to be called "tomcat_access_log.2010-09-08.txt". I'll end up with hundreds more sources. I'd rather it simply be called "tomcat_access_log" or something like that.

So let me take a stab at this:

In inputs.conf:

[monitor:///tomcat/logs/tomcat_access_log*]
sourcetype = tomcat-access

In props.conf:

[tomcat-access]
TRANSFORMS-rename_source = drop_trailing_text

In transforms.conf:

[drop_trailing_text]
DEST_KEY   = MetaData:Source
SOURCE_KEY = MetaData:Source
REGEX    = source::(.*)\d\d\d\d-\d\d-\d\d.txt
FORMAT   = source::$1

I guess it's the REGEX I'm uncertain about.

Am I on the right track?

Thanks!

Tags (2)
0 Karma
1 Solution

Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

View solution in original post

Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

View solution in original post

Builder

Worked GREAT, thanks!

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!