Hi, I am using Splunk 6.5.
How can I exclude lines containing a pattern from being indexed? In my case I have IIS access logs forwarded by a Universal Forwarder. I have tried to configure like this, but log lines that contains bigip is still indexed.
It isn't possible to filter data using a Universal Forwarder, with the exception of Blacklisting or Whitelisting Wiindows event codes. You would need to use the props and transforms settings on the indexers, or use a Heavy Forwarder.
If using a Heavy Forwarder, you need to consider how much of the data you are actually filtering out. If it isn't a large percentage, then it isn't worth it since Heavy Forwarders send what is called "cooked data" which is larger than what a Universal Forwarder would send. So you really wouldn't be cutting back on any network traffic.
If you aren't filtering a large portion, use the Universal Forwarder and add the props.conf and transforms.conf settings to your indexers.