Hunk App for MongoDB: How to filter data coming in from MongoDB before getting indexed (virtual index)?

[prachisaxena]

Hi All,

I have a MongoDB and Splunk Enterprise setup with a trial license for Hunk.
Have created a Virtual Index and Provider using Hunk App for MongoDB and it is working fine.
The data in MongoDB is huge and we do not want to get all of that data into Splunk Enterprise because it is getting charged under the Splunk Enterprise License.

Is there a way to filter the data coming in from MongoDB before it is getting indexed through Splunk Indexer.

[rdagan_splunk · Accepted solution]

You have two main options

1) The first option is to filter based on Time:
If you setup time field in the configurations, you can use the Hunk Time picker to bring less data based on time. For example, Query Database = A, Table = B, For only 3 days worth to data
vix.mongodb.db = hunk
vix.mongodb.collection = test
vix.mongodb.field.time = _id
vix.mongodb.field.time.format = ObjectId

2) The second option is to filter based on Key Value pair:
In this example, only foo=xyz dataset will be brought back from MongoDB
index=mongodb foo=xyz | timechart avg(bar) by baz

[kartik13]

You can black list the data in inputs.conf or even can parse the data using Heavy forwarder or even writing a script for parsing or if you want cheap-fix you can decrease the throughput .It will reduce the speed for ingestion of data in splunk which will be under scheme.

[prachisaxena]

Hi Karthik,

Thanks for helping.. I was wondering if we can use inputs.conf, props.conf and transforms.conf in a similar way as we use on Splunk Enterprise filtering. The requirement is it filter the data on MongoDB app for hunk and currently it is not using any inputs.conf. I am still newbie to splunk to may be i am wrong here