All Apps and Add-ons

Splunk Add-on for IBM WebSphere Application Server: Why is filtering events to nullQueue not working with WAS logs?

dschmidt_cfi
Path Finder

I have reviewed and tried most ever suggestion that I have seen on this site but still no luck. I am trying to filter out, pre-index, all java stack traces containing lines like robots.txt, favicon.ico, etc. These are WebSphere 8 Application Server logs and I am currently testing this in my sandbox. I am using the Splunk_TA_ibm-was which has a sourcetype of ibm:was:systemOutLog for the SystemOut.log

As I mentioned I have tried several variations that all work on the search command line like:

sourcetype="ibm:was:systemOutLog" | REGEX _raw != "(/apple.+png|/favicon.ico|/robots.txt|/yahoo-dom-event.js)"

Which reduces the total number of events from 58,785 to 33,303. Below are my last attempt's configuration:

props.conf

[sourcetype::ibm:was:systemOutLog]
TRANSFORMS-null = null_queue_filter

transforms.conf

[null_queue_filter]
REGEX=(/apple.+png|/favicon.ico|/robots.txt|/yahoo-dom-event.js)
DEST_KEY=queue 
FORMAT=nullQueue

I have tried these in several places, but I believe that /opt/splunk/etc/apps/Splunk_TA_ibm-was/local/ is the correct location. I leave these in the web server logs, but do not need the stack traces that java dumps on everything. All applications are running under RHEL 6 if that makes a difference.

Just in case;
ibm_was.conf
(one of the four entries)

[monitor:///opt/IBM/WebSphere/AppServers/profiles/DMT-AS8P03/logs]
 whitelist = SystemOut.log
 crcSalt = <SOURCE>
 disabled = false
 followTail = false
 index = cfnc_appsrv
 host =
 host_segment = 6
 sourcetype = ibm:was:systemOutLog

TIA as I am sure it is something simple I am overlooking.

0 Karma
1 Solution

somesoni2
Revered Legend
  1. The stanza name for props.conf is wrong. For sourcetypes, you just specify the name. Replace [sourcetype::ibm:was:systemOutLog] with just [bm:was:systemOutLog]
  2. The props and transforms should be in Indexer/Heavy forwarder, preferably under an app.

View solution in original post

somesoni2
Revered Legend
  1. The stanza name for props.conf is wrong. For sourcetypes, you just specify the name. Replace [sourcetype::ibm:was:systemOutLog] with just [bm:was:systemOutLog]
  2. The props and transforms should be in Indexer/Heavy forwarder, preferably under an app.

dschmidt_cfi
Path Finder

Unbelievable simple mistake by me, but you were correct. Thank you. Now to calculate the impact against our license in the filtered state and wait for approval to add.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...