Getting Data In

Multiple sourcetypes in the same directory

jheilman
Explorer

I know this question has been asked numerous times before, because I've read most of the questions and answers. I still can't seem to get it right, no matter what I try. We have several Windows servers running JBoss. The folder structure is similar to the following...

D:\jboss\server\<ContainerName>\log\
    access.2011-05-24.log
    app.log
    boot.log
    stderr.log
    stdout.log

So, the goal is to pull access.YYYY-MM-DD.log as sourcetype=access_common and everything else as sourcetype=log4j. Ideally, I'd like to be able to create a JBoss server class and push a generic configuration out to all of our JBoss servers to pull the logs.

I've tried several different things, but nothing seems to work as expected. I've tried using simple regular expressions in the [monitor] stanzas as suggested in one answer and I've tried a very general [monitor:] stanza pointing at the directory with accompanying [source::] stanzas to filter the file names and specify sourcetypes in props.conf. I've tried more than that, but those two seemed to be the most promising. I've used https://servername:8089/services/admin/inputstatus/TailingProcessor:FileStatus to verify the files are being read, but they don't seem to be getting indexed, or they don't have the expected sourcetype if they are.

I know things have changed from version to version in Splunk and maybe the problem is that I'm trying things that don't work anymore. Can someone set me straight?

My current configuration is as follows...

inputs.conf:

#
# JBoss - Common Log Files
#
[monitor://D:\jboss\server\*\log\*.log]
index = fod-web

props.conf:

[source::...\\access\.\d{4}-\d{2}-\d{2}\.log$]
sourcetype = access_common

[source::...\\(?!access)[\w-_.]+\.log$]
sourcetype = log4j

jbsplunk
Splunk Employee
Splunk Employee

Could you paste the configuration you are using to try and do your sourcetyping? Also, you may want to review the following, there are some pretty good example configurations:

http://blogs.splunk.com/2010/02/11/sourcetypes-gone-wild/

I think you should be using:

[source::…\access.\d{4}-\d{2}-\d{2}.log$]

and

[source::...\(?!access)[\w+.log$]

0 Karma

jheilman
Explorer

I updated the original question with the configuration I'm using currently.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...