topic Re: Only index events that match criteria in Getting Data In

Only index events that match criteria

cdevoe57 — Fri, 17 Oct 2025 23:58:16 GMT

I am using the TA_nix addon to get information about the services on the servers. I am only interested in indexing 10 of the 150 services.

My setup is as follows

Server UF --> HF --> 3 node index cluster

I am trying to filter out the events at the HF

These are the props and transforms files

props.conf

[source :: Unix:Service]
TRANSFORMS-set= setnull,setparsing

transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \[sshd\]
DEST_KEY = queue
FORMAT = indexQueue

Eventually I will set it to allow several services, sshd, auditd, etc.

I'm not sure what I am doing wrong here or missing. But this is not working

Re: Only index events that match criteria

livehybrid — Sat, 18 Oct 2025 10:06:54 GMT

Hi @cdevoe57

Can I check where you have applied the props/transforms?

In this setup it should be applied to the HF.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Only index events that match criteria

PickleRick — Sat, 18 Oct 2025 16:50:11 GMT

I don't recall the details of the TA_nix but I'm pretty sure that

[source :: Unix:Service]

is wrong.

Firstly, there shouldn't be spaces in the stanza name. But that might just be a copy-pasting error.

Secondly, if I remember correctly, you get a script name as source. Unix:Service looks more like a sourcetype name (but I don't recall such sourcetype either). How did you come up with this?

Re: Only index events that match criteria

richgalloway — Sat, 18 Oct 2025 18:09:08 GMT

I suggest using Ingest Actions (IA) to filter out the unwanted events. IA uses transforms, but has an easy-to-use UI that lets you test your regular expressions before submitting them.

You can implement IA on your HF or indexers if the GUI is enabled). If the GUI is not enabled, set up the rules on a search head and copy the resulting transforms (they will be 'ruleset' settings) to the HF.

Re: Only index events that match criteria

PickleRick — Sun, 19 Oct 2025 06:56:12 GMT

Just to nitpick a little. Ingest Actions are a wrapper for RULESET functionality, not TRANSFORMS.

Re: Only index events that match criteria

cdevoe57 — Mon, 20 Oct 2025 06:20:43 GMT

It is being applied on the HF

Re: Only index events that match criteria

cdevoe57 — Mon, 20 Oct 2025 06:22:28 GMT

I tried it with and without the spaces. I will remove them.

As for the Unix:Service I got that by doing a query and looking at the source.

Re: Only index events that match criteria

PickleRick — Mon, 20 Oct 2025 13:00:20 GMT

Yeah, I rechecked and there is indeed that source (and sourcetype) for the service.sh scripted input.

But there definitely should _not_ be a space there.

Also - if you look into the default props.conf supplied by the addon, matching to source is done by

[source::...Unix:Service]

I'm not sure why, to be honest.

Re: Only index events that match criteria

cdevoe57 — Mon, 20 Oct 2025 14:51:04 GMT

That in fact worked. Thanks.

Now to get the regex

Re: Only index events that match criteria

cdevoe57 — Tue, 21 Oct 2025 14:22:14 GMT

In the end this is what worked

props.conf

## Version 10/20/2025 16:00

[source::...Unix:Service]
TRANSFORMS-set= setnull,setparsing

transforms.conf

## Version 10/20/2025 16:00

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
## The list of services to index in alphabetical order

REGEX=auditd\.service|iptables\.service|journald\.service|nftables\.service|rsylog\.service||sysstat\.serviceDEST_KEY = queue
FORMAT = indexQueue