i was tyring to filter a set of data to indexer by filtering out few data and below are the sample logs and configurations:
Here trying to pass only category_id=FLOWERS to the indexer and ignore GIFTS events.
sample log:
177.23.21.50 - - [24/Jul/2014:03:42:00] "GET /flower_store/category.screen?category_id=GIFTS HTTP/1.1" 200 10591 "http://mystore.splunk.com/flower_store/main\\.screen&JSESSIONID=SD2SL2FF7ADFF5" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 2035 1226
233.77.49.46 - - [24/Jul/2014:03:41:46] "GET /flower_store/product.screen?product_id=K9-BD-01 HTTP/1.1" 200 10560 "http://mystore.splunk.com/flower_store/category.screen?category_id=GIFTS&JSESSIONID=SD2SL2FF7ADFF5" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 661 1822
177.23.21.50 - - [24/Jul/2014:03:42:00] "GET /flower_store/category.screen?category_id=FLOWERSHTTP/1.1" 200 10591 "http://mystore.splunk.com/flower_store/main\\.screen&JSESSIONID=SD2SL2FF7ADFF5" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 2035 1226
Configuration:
inputs.conf
[monitor:///opt/log/willwork.log]
sourcetype = access_common
index=heavy
outputs.conf
[tcpout]
defaultGroup = my_search_peers
forwardedindex.filter.disable = true
indexAndForward = false
[tcpout:my_search_peers]
server=indexerip:9997
[monitor:///opt/log/willwork.log]
sourcetype = access_common
index=heavy
props.conf
[access-combine]
TRANSFORMS-routing=accessrouting
transforms.conf
[accessrouting]
REGEX=FLOWERS
DEST_KEY=_TCP_ROUTING
FORMAT=my_search_peers
data is getting indexer but GIFTS even is also getting indexed
