Splunk Search

Is there a way I can see how much data is being searched per index?

ssagar1009
New Member

Is there a way I can see how much data is being searched per index?

Eg: for an index, a user has searched 10 GB of data over the last 1 hour in across 15 search queries.

An index has 100 GB of data, but the last 1-day user searched only 100 MB in the search result.

or Index has 100 GB of data, but user searched too often and search a total of 120 GB of data.

0 Karma

woodcock
Esteemed Legend

I can tell you generally how to do this, but it seems crazy to me and there doesn't seem to be any useful reason to do this. You start by use a REST query to tell you what searches have run recently. Then you query the job details REST API to get access to the search.log for the job. In there it will tell you the number of events scanned. You can then get the optimized search from the same log and hopefully get a sourcetype from there. Then you can perform a search using avg(len(_raw) against that sourcetype and multiply this iw the events scanned number. That is probably the best that you can do it is making many assumptions and not going to be too accurate.

0 Karma

HiroshiSatoh
Champion

Please execute the following search sentence. Only inaccurate information on this level can be acquired.

I do not know the purpose, but I recommend you to review alternative proposals.

index=_audit action=search search=* sourcetype=audittrail    
| rex field=search "index\s*=\s*\"*(?<IndexUsed>[^\s\"]+)" 
| search IndexUsed=*  
| fillnull value="NA" IndexUsed 
| stats count  by IndexUsed
0 Karma

ssagar1009
New Member

Above query doesn't give the size of data searched per index, it just shares count on number time index used.

Purpose is to validate, data stored vs data actually searched. This is to check abuse on capacity utilization.
Any other recommendation will be welcome!
Thanks

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...