Splunk Search

Tomcat log rotation question

Branden
Builder

I read a similar post about this here but I'm not sure I completely understand.

My tomcat log name looks like this: tomcat_access_log.2010-09-08.txt

I understand I can just monitor for /tomcat/logs/tomcat_access_log* (right?)

When it gets indexed, I don't want the source to be called "tomcat_access_log.2010-09-08.txt". I'll end up with hundreds more sources. I'd rather it simply be called "tomcat_access_log" or something like that.

So let me take a stab at this:

In inputs.conf:

[monitor:///tomcat/logs/tomcat_access_log*]
sourcetype = tomcat-access

In props.conf:

[tomcat-access]
TRANSFORMS-rename_source = drop_trailing_text

In transforms.conf:

[drop_trailing_text]
DEST_KEY   = MetaData:Source
SOURCE_KEY = MetaData:Source
REGEX    = source::(.*)\d\d\d\d-\d\d-\d\d.txt
FORMAT   = source::$1

I guess it's the REGEX I'm uncertain about.

Am I on the right track?

Thanks!

Tags (2)
0 Karma
1 Solution

Lowell
Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

View solution in original post

Lowell
Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

Branden
Builder

Worked GREAT, thanks!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...