Splunk Search
Highlighted

Unable to nullqueue unnecessary lines without a date

Path Finder

I have two lines of events that are unnecessary because there is no date and would like to null queue these out. I have tried several REGEX statements ( I have tried ^\s+(\<\/ns2:Subject.*\>), <\/ns2:.*\>, and to name a few) to delete these events. The Field Extraction GUI and REGEX online tester show a match. However, these events still show up in the search.
The data which needs to be deleted are as follows:
(Please note there are spaces showing before the events and they are on separate lines)

 </ns2:Subject>
       </ns2:SubjectConfirmation>

transforms.conf

[TrashEmptySubject]
REGEX = ^\s+(\<\/ns2:Subject.*\>)
DEST_KEY = queue

props.conf

[smtrace]
TRANSFORMS-null = TrashEmptySubject
FORMAT = nullQueue
0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Contributor

add FORMAT = nullQueue to transforms.conf, instead of props.conf

0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Esteemed Legend

Try this:

props.conf

[smtrace]
TRANSFORMS-null = TrashEmptySubject

transforms.conf

[TrashEmptySubject]
REGEX = [\r\n\s]+<\/ns2:Subject[^>]+>
DEST_KEY = queue
FORMAT = nullQueue
0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Path Finder

Thank you for your responses. Unfortunately, the events are still showing when I do a search after applying this REGEX

0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Esteemed Legend

If you are doing a sourcetype override/overwrite, you must use the ORIGINAL value NOT the new value, then you must deploy this to the first full instance(s) of Splunk that handles the events (usually either the HF-tier, if you use this, or your Indexer tier), restart all Splunk instances there, send in new events (old events will stay broken), then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly indexed events.

0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Path Finder

Thank you for the help. We are not doing any sourcetype overide. We are getting these events directly from a forwarder, passing through one of two HF's (not being indexed), ultimately landing on a Splunk instance which is both search head / indexer. I have edited both the HF's and search head / indexer with these stanzas and we are still seeing these type of events.

0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Esteemed Legend

My answer definitely should work. Did you notice that you had a line in props.conf that should be in transforms.conf?

0 Karma
Highlighted

Re: Unable to nullqueue unnecessary lines without a date

Path Finder

Here are my configurations:

props.conf
[smtrace]
TRANSFORMS-null = TrashEmptySubject

transforms.conf
[TrashEmptySubject]
REGEX = [\r\n\s]+<\/ns2:Subject[^>]+>
DEST_KEY = queue
FORMAT = nullQueue

0 Karma