exclude events based on field

[martinnepolean]

Hi,
Using filemonitor. we are collecting data from a file which sends data of all nix servers. Now we want to only exclude the linux servers. One of the field in teh events have the Ip address of the destination linux servers and we can use it differentiate the servers. But I am not sure how and where I have to configure this blacklist.

[richgalloway]

Filter event using transforms. This is impractical for a long list of addresses, however.

Props.conf:

[mysourcetype]
TRANSFORMS-filter = filterLinux

Transforms.conf:

[filterLinux]
# Enter Linux IP addresses here
REGEX = ipAddress = (10\.1\.2\.3|10\.2\.3\.4|10\.3\.4\.5)
DEST_KEY = nullQueue

[martinnepolean]

We already have props and transforms on this filemonitor forwarder to change the index based on the event. so where I have to add the above filter? forwarder or indexer?

[richgalloway]

The settings in my answer go on your indexers or heavy forwarders, whichever is first to process the events.

I'm not aware of a method to look up host names at index time. If such a method exists, it would slow indexing significantly.

Is it possible to change how the data is logged? Perhaps add a platform/OS indication? Maybe separate the data into separate files by platform?

[martinnepolean]

And the IP address list is big, is it possible
1. To get the hostname of those IP in the new field(maybe run nslookup and assign it to the new field)
2. and use them for filtering because we need not be updating this file if new Linux server comes into the environment.

[richgalloway]

To clarify:
You have a file containing events from several systems. [Why not have each system send to Splunk?]
You want to exclude the events from Linux systems.
Linux systems are identified by IP address.
Do you want to exclude the data at index time or search time? Doing so at index time may be a challenge unless the list of Linux IP addresses is short and static.

[martinnepolean]

1. These are particular application log on all servers forwarded to the application log server.
2. Yes, I want to exclude only linux events
3. all servers logs are identified by IP address only
4. i want to exclude @ index time. we have the list of IP which need to be excluded.

looking for a way to exclude events based on a field which has Ip address