topic Re: How to ensure regex filters in transforms.conf and a stanza in props.conf only applies to a specific input? in Getting Data In

How to ensure regex filters in transforms.conf and a stanza in props.conf only applies to a specific input?

bms9nmh — Mon, 05 Aug 2019 17:44:38 GMT

Hello, so I understand that my props.conf and transforms.conf (below) in theory allow me to filter out the events that match the regex specified.

props.conf

 [filter_out_auth_logs]
 TRANSFORMS-tonull = filter_out_word

transforms.conf

 [filter_out_word]
 REGEX =  WORD\[.*?\]:
 DEST_KEY = queue
 FORMAT = nullQueue

What I am unsure of, is how I ensure this filter is only applied to a specific input?

For example, if I have the following entries in the input.config file, where do I specify that I want the input that sends logs to myindex1 to make use of the filtering specified in props and transforms configs?:

 [monitor:///var/log/syslog]
    index = myindex1
    sourcetype = syslog

[monitor:///var/log/syslog.log]
        index = myindex2
        sourcetype = syslog

Re: How to ensure regex filters in transforms.conf and a stanza in props.conf only applies to a specific input?

audunn — Mon, 05 Aug 2019 18:55:34 GMT

You can do this by specifying the source to have action taken against in props.conf. So you'll end up with a props.conf config that looks something like this:

[source::/var/log/syslog]
REGEX = WORD[.*?]:
DEST_KEY = queue
FORMAT = nullQueue

You want to set the name of your props.conf stanza to refer to some metadata which splunk can use to process the logs through your transforms.

You can find more about the specifics here: https://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf
(note: []
* This stanza enables properties for a given .
* A props.conf file can contain multiple stanzas for any number of
different .
* Follow this stanza name with any number of the following setting/value
pairs, as appropriate for what you want to do.
* If you do not set a setting for a given , the default is used.

can be:
1. , the source type of an event.
2. host::, where is the host, or host-matching pattern, for an
event.
3. source::, where is the source, or source-matching
pattern, for an event.
4. rule::, where is a unique name of a source type
classification rule.
5. delayedrule::, where is a unique name of a delayed
source type classification rule.
These are only considered as a last resort
before generating a new source type based on the
source seen.)

Re: How to ensure regex filters in transforms.conf and a stanza in props.conf only applies to a specific input?

bms9nmh — Mon, 05 Aug 2019 19:11:38 GMT

Hello, thanks for the response. Adding [source::/var/log/syslog]
in the stanza won't help for my use case. I have different inputs that send logs to multiple indexes, but they have the same source log, and I want to apply the filter to one of these inputs, but not the other. Is there anything else I can do?
Is there no way to point the input at the props stanza?

Re: How to ensure regex filters in transforms.conf and a stanza in props.conf only applies to a specific input?

audunn — Mon, 05 Aug 2019 19:25:48 GMT

how we handle that in our org is by giving those a unique sourcetype and stitching everything together for end user simplicity with tags