I have reviewed and tried most ever suggestion that I have seen on this site but still no luck. I am trying to filter out, pre-index, all java stack traces containing lines like robots.txt, favicon.ico, etc. These are WebSphere 8 Application Server logs and I am currently testing this in my sandbox. I am using the Splunk_TA_ibm-was which has a sourcetype of ibm:was:systemOutLog for the SystemOut.log
As I mentioned I have tried several variations that all work on the search command line like:
sourcetype="ibm:was:systemOutLog" | REGEX _raw != "(/apple.+png|/favicon.ico|/robots.txt|/yahoo-dom-event.js)"
Which reduces the total number of events from 58,785 to 33,303. Below are my last attempt's configuration:
[sourcetype::ibm:was:systemOutLog] TRANSFORMS-null = null_queue_filter
[null_queue_filter] REGEX=(/apple.+png|/favicon.ico|/robots.txt|/yahoo-dom-event.js) DEST_KEY=queue FORMAT=nullQueue
I have tried these in several places, but I believe that /opt/splunk/etc/apps/Splunk_TA_ibm-was/local/ is the correct location. I leave these in the web server logs, but do not need the stack traces that java dumps on everything. All applications are running under RHEL 6 if that makes a difference.
Just in case;
(one of the four entries)
[monitor:///opt/IBM/WebSphere/AppServers/profiles/DMT-AS8P03/logs] whitelist = SystemOut.log crcSalt = <SOURCE> disabled = false followTail = false index = cfnc_appsrv host = host_segment = 6 sourcetype = ibm:was:systemOutLog
TIA as I am sure it is something simple I am overlooking.
Unbelievable simple mistake by me, but you were correct. Thank you. Now to calculate the impact against our license in the filtered state and wait for approval to add.