Location of props.conf and transforms.conf in a distributed setup

[michaelnorup]

Hi Guys.
I have a distributed setup consisting of 1 search head, 1 deployment/license server, 1 indexer.
And a whole bunch of universal forwarders.

I am trying to filter out some of the data coming in with transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = "name":"System availability"
DEST_KEY = queue
FORMAT = indexQueue

and a props.conf

[Zabbix-history]
SHOULD_LINEMERGE = falde
MAX_TIMESTAMP_LOOKAHEAD = 300
detect_trailing_nulls = auto
TIME_PREFIX = \"clock\":
KV_MODE = json
AUTO_KV_JSON = true
TRANSFORMS-set = setnull,setparsing

A log example that i would like to index, matching the regex in transforms.conf

{"host":{"host":"xxxxx","name":"xxxx"},"groups":["xxxx Prod","xxxx","Windows servers"],"item_tags":[{"tag":"SAP Basis","value":""},{"tag":"System availability","value":""},{"tag":"SID1","value":""},{"tag":"Product","value":"Web Server"},{"tag":"SID","value":"WSP"}],"itemid":900162,"name":"System availability","clock":1670486400,"count":13,"min":1,"avg":1,"max":1,"type":3}

Currently the props.conf and transforms.conf are on the indexer in the designated app.

Its currently filtering out all the logs with the sourcetype Zabbix-history, and not indexing the "name":"System Availability"

Does the props/transforms also need to be on the searchhead, or pushed to the universalforwarder with the deployment server?

[gcusello · Accepted solution]

Hi @michaelnorup,

filtering action is runned only on Indexers or (when present) on Heavy Forwarders, so if you have only filtering configurations you don't need to put your props.conf and transpose.conf files on Search Head or Universal Forwarders (via Deployment Server).

But if in the same props.conf and transformas.conf there are other parsing settings (e.g. fied extractions, lookups definiton, etc...) in this case you have to put the conf files also in Search Head.

A least, if in the props.conf there's also the INDEXED_EXTRACTIONS option, you have also to put conf files also on Universal Forwarders (via DS).

Ciao.

Giuseppe

[gcusello]

Hi @michaelnorup,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉