Splunk Search

Tomcat log rotation question

Branden
Builder

I read a similar post about this here but I'm not sure I completely understand.

My tomcat log name looks like this: tomcat_access_log.2010-09-08.txt

I understand I can just monitor for /tomcat/logs/tomcat_access_log* (right?)

When it gets indexed, I don't want the source to be called "tomcat_access_log.2010-09-08.txt". I'll end up with hundreds more sources. I'd rather it simply be called "tomcat_access_log" or something like that.

So let me take a stab at this:

In inputs.conf:

[monitor:///tomcat/logs/tomcat_access_log*]
sourcetype = tomcat-access

In props.conf:

[tomcat-access]
TRANSFORMS-rename_source = drop_trailing_text

In transforms.conf:

[drop_trailing_text]
DEST_KEY   = MetaData:Source
SOURCE_KEY = MetaData:Source
REGEX    = source::(.*)\d\d\d\d-\d\d-\d\d.txt
FORMAT   = source::$1

I guess it's the REGEX I'm uncertain about.

Am I on the right track?

Thanks!

Tags (2)
0 Karma
1 Solution

Lowell
Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

View solution in original post

Lowell
Super Champion

It looks like the shown config should work. However, you should probably replace ".txt" with "\.txt$" in your regex.

Keep in mind that this will only effect newly indexed data. Any previously loaded content will not be changed unless you dump all your indexed data and reload. (You will have to decided if it's worth that kind of effort to have nice source names.)

BTW, I've written a regex to handle this kind of source transformation if you want to use it. It's a bit more flexible and it's been tested. See the transforms.conf stanza called source_clean-YYYY-MM-DD example from this post: http://answers.splunk.com/questions/3470/consolidate-similarly-named-log-files-into-a-single-source/...

Branden
Builder

Worked GREAT, thanks!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...