Hello All,
I have requirement where need to filter(ignore) "---------------------------------------------" from the event logs, I have tried with the blacklist attribute in inputs.conf, but it is not working. Do we need to create props.conf and transforms.conf file, or we can ignore these dashes from inputs.conf only?
Please let me know which solution is best here, if we need to create a props.conf
and transforms.conf
, then what will be the contents of the files?
Thanks
Hey!
First off these configurations happen in the parsing phase, before the indexing and searching phase, therefore
both props.conf and transforms.conf should be placed in the indexer, not the search head as you mentioned before. My suggestion is that you place these files in $SPLUNK_HOME/etc/system/local
The sourcetype stanza in props.conf isn't right, it should be as follows:
props.conf
[mydata]
TRANSFORMS-null = setnull
Your regex is not matching the repeated dashes in the event, so I put together a little regex for you.
transforms.conf
[setnull]
REGEX = ^.*\s(\-+\S)
DEST_KEY = queue
FORMAT = nullQueue
Restart Splunk after changing these files.
If you don't wanna have the wrong old indexed data with the repeated dashes, one option is to use the DELETE command in a search query that matches those events with "------" in them. i.e:
index=yourindex sourcetype=mydata "---------------------------------------------" | DELETE
This will hide the matched events from further searches but will not erase them completely from the index.
Splunk by default doesn't allow users to run the command DELETE, so go to "Access controls » Roles » "
and apply the role "can_delete"
to your user.
Hope this works for you 😃
/Santiago
Hello Woodcock,
Thanks for reply, as suggested I have deployed props.conf and transforms.conf on search heads but still the events are not filtering please let me know where i went wrong.
Took sourcetype instate of source
Props.conf
[sourcetype::mydata]
TRANSFORMS-null= setnull
Transforms.conf
[setnull]
REGEX = [^----------------------------------------]
DEST_KEY = queue
FORMAT = nullQueue
You need to deploy these to your Indexers (not your Search Head) and then restart the Splunk instances running on them. After that, the new data will be fixed but the old/existing data will still be "wrong". Also, use this (no square brackets) instead of what you have:
REGEX = ^----------------------------------------
Don't forget to "Accept" and answer to close the question.
Did this work?