Splunk Search

Tagging based on source

1StopBloke
Explorer

Hello,

We use log4net for a bunch of our windows services and web applications. Currently I set the sourcetype for each of the log4net logs using props.conf based on the source file and I make it a meaningful name like sourcetype=log4net:service:dispatcher or sourcetype=log4net:web:portal, something along those lines. The log files are rolled daily, so they have the date appended to the log file each day. e.g.

ERROR\_service\_dispatcher.20120423.log
ERROR\_service\_dispatcher.20120424.log

I use wildcards in my props.conf to match the source due to the date rolling. We also have multiple hosts that use the same log format and applications. So three hosts or so could have the same log file names above, etc.

As almost all of the log4net logs are a common format I'd like to change the sourcetype to log4net, which let's me easily put in a bunch of field extractions and a few other things, and then just tag the logs with the meaningful information rather than having it in the sourcetype. This seems like a better way to search for things as well as the other advantates.
I was envisioning having tags in key value pairs, so I'd have something like

log4netfamily=service or log4netfamily=web

then drilling down into the actual application names, like

log4netapp=dispatcher or log4netapp=portal

As far as I can tell there is no way to do this easily with tags except by mucking about with your props and transforms and appending a key value string to your log entry each time it gets indexed. Seems like a hassle, no? What's the best way to achieve having additional key value tags (essentially an extra field) based on source?

I was also thinking having single value tags would be okay too (tag=log4netservice or tag=log4netweb is fine with me), but I'm having problems getting that going too.
What I was hoping I could do is just add a source stanza to my tags.conf based on the file name, so for example in tags.conf

[source=.*log4net[/\\]ERROR\_service\_dispatcher\.\d+\.log] 
log4netservice = enabled 
log4netdispatcher = enabled

(I also tried the following alternative on the tag, I'm not sure which is correct)

[source=*ERROR\_service\_dispatcher.*.log]

But neither seems to apply the tags (I'm searching for tag::source="log4netservice", is that right?). From what I can see/read it seems wildcards don't work in tags.conf. Is that so? What are my options here?

If this all seems a bit pointless I'll just put in a wildcard extraction stanza as per this splunkbase article, but it seems like a bit of a no no based on the comments:
http://splunk-base.splunk.com/answers/8505/is-it-possible-to-use-wildcards-in-sourcetype-propsconf-s...

0 Karma

1StopBloke
Explorer

Any ideas?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2026-2027 SplunkTrust is officially open. If ...