Getting Data In

Multiple sourcetypes in the same directory

jheilman
Explorer

I know this question has been asked numerous times before, because I've read most of the questions and answers. I still can't seem to get it right, no matter what I try. We have several Windows servers running JBoss. The folder structure is similar to the following...

D:\jboss\server\<ContainerName>\log\
    access.2011-05-24.log
    app.log
    boot.log
    stderr.log
    stdout.log

So, the goal is to pull access.YYYY-MM-DD.log as sourcetype=access_common and everything else as sourcetype=log4j. Ideally, I'd like to be able to create a JBoss server class and push a generic configuration out to all of our JBoss servers to pull the logs.

I've tried several different things, but nothing seems to work as expected. I've tried using simple regular expressions in the [monitor] stanzas as suggested in one answer and I've tried a very general [monitor:] stanza pointing at the directory with accompanying [source::] stanzas to filter the file names and specify sourcetypes in props.conf. I've tried more than that, but those two seemed to be the most promising. I've used https://servername:8089/services/admin/inputstatus/TailingProcessor:FileStatus to verify the files are being read, but they don't seem to be getting indexed, or they don't have the expected sourcetype if they are.

I know things have changed from version to version in Splunk and maybe the problem is that I'm trying things that don't work anymore. Can someone set me straight?

My current configuration is as follows...

inputs.conf:

#
# JBoss - Common Log Files
#
[monitor://D:\jboss\server\*\log\*.log]
index = fod-web

props.conf:

[source::...\\access\.\d{4}-\d{2}-\d{2}\.log$]
sourcetype = access_common

[source::...\\(?!access)[\w-_.]+\.log$]
sourcetype = log4j

jbsplunk
Splunk Employee
Splunk Employee

Could you paste the configuration you are using to try and do your sourcetyping? Also, you may want to review the following, there are some pretty good example configurations:

http://blogs.splunk.com/2010/02/11/sourcetypes-gone-wild/

I think you should be using:

[source::…\access.\d{4}-\d{2}-\d{2}.log$]

and

[source::...\(?!access)[\w+.log$]

0 Karma

jheilman
Explorer

I updated the original question with the configuration I'm using currently.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...