Getting Data In

Location of props.conf and transforms.conf in a distributed setup

michaelnorup
Communicator

Hi Guys.
I have a distributed setup consisting of 1 search head, 1 deployment/license server, 1 indexer.
And a whole bunch of universal forwarders.

I am trying to filter out some of the data coming in with transforms.conf:

 

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = "name":"System availability"
DEST_KEY = queue
FORMAT = indexQueue

 

 


and a props.conf

 

[Zabbix-history]
SHOULD_LINEMERGE = falde
MAX_TIMESTAMP_LOOKAHEAD = 300
detect_trailing_nulls = auto
TIME_PREFIX = \"clock\":
KV_MODE = json
AUTO_KV_JSON = true
TRANSFORMS-set = setnull,setparsing

 

 


A log example that i would like to index, matching the regex in transforms.conf

 

{"host":{"host":"xxxxx","name":"xxxx"},"groups":["xxxx Prod","xxxx","Windows servers"],"item_tags":[{"tag":"SAP Basis","value":""},{"tag":"System availability","value":""},{"tag":"SID1","value":""},{"tag":"Product","value":"Web Server"},{"tag":"SID","value":"WSP"}],"itemid":900162,"name":"System availability","clock":1670486400,"count":13,"min":1,"avg":1,"max":1,"type":3}

 

 

 

Currently the props.conf and transforms.conf are on the indexer in the designated app.

Its currently filtering out all the logs with the sourcetype Zabbix-history, and not indexing the "name":"System Availability"

Does the props/transforms also need to be on the searchhead, or pushed to the universalforwarder with the deployment server?

Labels (3)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @michaelnorup,

filtering action is runned only on Indexers or (when present) on Heavy Forwarders, so if you have only filtering configurations you don't need to put your props.conf and transpose.conf files on Search Head or Universal Forwarders (via Deployment Server).

But if in the same props.conf and transformas.conf there are other parsing settings (e.g. fied extractions, lookups definiton, etc...) in this case you have to put the conf files also in Search Head.

A least, if in the props.conf there's also the INDEXED_EXTRACTIONS option, you have also to put conf files also on Universal Forwarders (via DS).

Ciao.

Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @michaelnorup,

filtering action is runned only on Indexers or (when present) on Heavy Forwarders, so if you have only filtering configurations you don't need to put your props.conf and transpose.conf files on Search Head or Universal Forwarders (via Deployment Server).

But if in the same props.conf and transformas.conf there are other parsing settings (e.g. fied extractions, lookups definiton, etc...) in this case you have to put the conf files also in Search Head.

A least, if in the props.conf there's also the INDEXED_EXTRACTIONS option, you have also to put conf files also on Universal Forwarders (via DS).

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @michaelnorup,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...