Splunk Search

Tomcat log rotation question

Branden
Builder

I read a similar post about this here but I'm not sure I completely understand.

My tomcat log name looks like this: tomcat_access_log.2010-09-08.txt

I understand I can just monitor for /tomcat/logs/tomcat_access_log* (right?)

When it gets indexed, I don't want the source to be called "tomcat_access_log.2010-09-08.txt". I'll end up with hundreds more sources. I'd rather it simply be called "tomcat_access_log" or something like that.

So let me take a stab at this:

In inputs.conf:

[monitor:///tomcat/logs/tomcat_access_log*]
sourcetype = tomcat-access

In props.conf:

[tomcat-access]
TRANSFORMS-rename_source = drop_trailing_text

In transforms.conf:

[drop_trailing_text]
DEST_KEY   = MetaData:Source
SOURCE_KEY = MetaData:Source
REGEX    = source::(.*)\d\d\d\d-\d\d-\d\d.txt
FORMAT   = source::$1

I guess it's the REGEX I'm uncertain about.

Am I on the right track?

Thanks!

Tags (2)
0 Karma
1 Solution

Lowell
Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

View solution in original post

Lowell
Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

Branden
Builder

Worked GREAT, thanks!

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...