Getting Data In

Applying different extractions to the same source from different hosts

krisreeves
Path Finder

I have two groups of servers that are both running haproxy, and the logs are in the same location (e.g. /var/log/haproxy.log). The log format is different for each of them. I'd like to specify to apply one REPORT transform to haproxy logs coming from *.foo.com and a different REPORT transform to haproxy logs coming from *.bar.com (search-time field extractions). I can't change the sourcetype name itself, it's expected to be 'haproxy' in both cases.

I can't determine any easy way to have a props.conf stanza that matches both a source AND a host, but it seems wasteful and slow to run two giant regular expressions for every line when only one needs to be applied.

Are there any config tricks that can be used here?

0 Karma

FrankVl
Ultra Champion

One trick can be to create a symlink that contains the server’s name (or some other string that helps you keep the two formats apart) which points tot the original log folder. Then point Splunk to the symlink (can still be a generic input stanza by using wildcard), such that the source value now becomes host specific.

We use this often to identify what syslog server a certain event passed through while keeping the original sending device in the host field.

0 Karma

ddrillic
Ultra Champion

It's interesting as you can use wildcards and regex for source and host stanzas in props.conf... but you need a combo ; - )

@MuS spoke to that at Can I have multiple specs (host, source, sourcetype) in props.conf?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

From Data to Insight: Announcing the Winners of the Splunk Dashboard Contest

Hi Splunkers, First off, thank you to everyone who participated in our very first From Data to Insight: The ...

Splunk Developers: Construct Your Future at the .conf26 Builder Bar

Calling all Splunk architects, platform admins, and app developers: the site is open, and the blueprints are ...

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...