Getting Data In

Multiple sourcetypes in the same directory

jheilman
Explorer

I know this question has been asked numerous times before, because I've read most of the questions and answers. I still can't seem to get it right, no matter what I try. We have several Windows servers running JBoss. The folder structure is similar to the following...

D:\jboss\server\<ContainerName>\log\
    access.2011-05-24.log
    app.log
    boot.log
    stderr.log
    stdout.log

So, the goal is to pull access.YYYY-MM-DD.log as sourcetype=access_common and everything else as sourcetype=log4j. Ideally, I'd like to be able to create a JBoss server class and push a generic configuration out to all of our JBoss servers to pull the logs.

I've tried several different things, but nothing seems to work as expected. I've tried using simple regular expressions in the [monitor] stanzas as suggested in one answer and I've tried a very general [monitor:] stanza pointing at the directory with accompanying [source::] stanzas to filter the file names and specify sourcetypes in props.conf. I've tried more than that, but those two seemed to be the most promising. I've used https://servername:8089/services/admin/inputstatus/TailingProcessor:FileStatus to verify the files are being read, but they don't seem to be getting indexed, or they don't have the expected sourcetype if they are.

I know things have changed from version to version in Splunk and maybe the problem is that I'm trying things that don't work anymore. Can someone set me straight?

My current configuration is as follows...

inputs.conf:

#
# JBoss - Common Log Files
#
[monitor://D:\jboss\server\*\log\*.log]
index = fod-web

props.conf:

[source::...\\access\.\d{4}-\d{2}-\d{2}\.log$]
sourcetype = access_common

[source::...\\(?!access)[\w-_.]+\.log$]
sourcetype = log4j

jbsplunk
Splunk Employee
Splunk Employee

Could you paste the configuration you are using to try and do your sourcetyping? Also, you may want to review the following, there are some pretty good example configurations:

http://blogs.splunk.com/2010/02/11/sourcetypes-gone-wild/

I think you should be using:

[source::…\access.\d{4}-\d{2}-\d{2}.log$]

and

[source::...\(?!access)[\w+.log$]

0 Karma

jheilman
Explorer

I updated the original question with the configuration I'm using currently.

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...