Trimming the amount of data forwarded

[tnconners]

Background: We have an existing indexer, that we have added a lot of data to. We would like to cut down on the amount of logs we index.

Basically, we alert on any logs that come in as HIGH or CRITICAL. I'm fairly certain I can set up the forwarder to only forward logs that have the word HIGH or CRITICAL. My question is how is this accomplished. I've looked through much of the documentation already and haven't found a clear cut answer.

Any help is appreciated!

EDIT: The indexer and Forwarders are both on windows, Splunk version is 5.05. I apologize but based on the nature of the logs I cannot post them. If you could give me a general view of how to do it, I could create the regex myself for it.

[lukejadamec]

So, basically you need to configure a nullQueue on the system that is cooking the data. Here is an example of dropping unwanted events from a windows security log:

Props.conf

[source::WMI:WinEventLog:Security]

TRANSFORMS-FilterEvent = FilterEvent560

Transforms.conf

[FilterEvent560]

REGEX = (?msi)^EventCode=560

DEST_KEY = queue

FORMAT = nullQueue

You will want to change the source in props.conf to match your source, and probably change the 'FilterEvent560' identifier to something that makes more sense to you, like 'dropLow'. You will also need to create a REGEX that matches what you want to drop.

If you have any questions let us know.

[tnconners]

To further my edit, the logs are actually pretty nice. There is some specific data then |high | or |critical | then more specific data.

[lukejadamec]

Yes, this can be done, but you need to provide more information.
An example of the logs for both with HIGH and CRITICAL and without, so we can actually give you specific answers would be nice.
What OS, and Splunk version?