Splunk Enterprise

behavior of props.conf

irwinj_125
Explorer

Hello,

Had a quick question with regards to props.conf and how it would behave.  We have a directory which has a large number of different logs and we use just one sourcetype for all (*.* in the path in inputs.conf).  

I am planning to setup the following props.conf for this sourcetype as the vast majority of the log files follow this date structure/setup.  However, a few of the logs do not.  I'm just wondering how these logs would behave?  Would they simply revert to the overall system default?  Of course I could setup separate sourcetypes for each file name if need be, but would rather continue with I have for now.

SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%d %H:%M:%S[\.,]%3N
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=24
BRE...

Thanks!

 

Labels (2)
0 Karma

ekenne06
Path Finder

I ran into this issue the other day. I ended following this document:

https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/Advancedsourcetypeoverrides

 

In summary what you'll do is  take the sourcetype you have now, and apply a transform. In that transform you specify a regex pattern that will specify what to use for sourcetype designation. Once you have that, you can either use the FORMAT command to specify the sourcetype like FORMAT = $1.$2. Once I find time, ill send an example from one of my instances. 

0 Karma

ekenne06
Path Finder

the data I have has a format like: Class.Method.Attribute, and I wanted the sourcetype to be SIXPACService:Method:Attribute

props.conf

[sixpac_logs]
TRANSFORMS-SIXPAC = sixpac_services
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}\d{2}:\d{2}:\d{2}:\.\d+
TIME_PREFIX = ^
SHOULD_LINEMERGE = False

transforms.conf

[sixpac_services]
REGEX = SIXPACService\.(\w+\.)\w+)\s
FORMAT = sourcetype::SIXPACService:$1:$2
DEST_KEY = MetaData:Sourcetype

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Events that don't comply with the defined props will inherit the timestamp from the previous event, which may or may not be good enough.

Using a single sourcetype for everything is optimism bordering on laziness.  😀  It's better for each distinct event format to have its own sourcetype (that's what source types are).  A little time spent up-front setting things up will make things easier and better later.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...