How do we measure Search Proficiency of our users?

[ddrillic]

We use the following in order to assess the search proficiency of our users -

-- Search Proficiency: A measure of how effectively saved searches are written by our users using our SPL language over the selected timeframe. It is calculated by measuring the number of events scanned (Scanned_Count) and the number of events brought back off disk (Event_Count). [Search Proficiency = (Event_Count / Scan_Count) * 100%]

Does it make sense? Can we improve on that?

[Richfez · Accepted solution]

I don't think that approach is nonsense and would tell you ... something, anyway.

Maybe combine that with some keyword analysis on the searches themselves?

This may get you started:

index=_audit action=search search=*
| rex field=search mode=sed "s/['`]//g"
| eval command_list = commands(search)
| chart count over command_list by user

I'd categorize the commands by "terribleness". We can probably help with that, but at this point I'll leave it as an exercise for the reader. 🙂

Some hints:
join is suspect every time it's used. The number of times I've seen join be legitimately needed is nearly zero - but it's common in some folk's SPL and a horrendously bad performing thing.

Some others that aren't necessarily wrong, but are good to sort of double-check
1- transaction usually can be replaced with stats, but sometimes it's the right thing.
2- append is sometimes OK, but can be misused....
3- streamstats can be awesome when used correctly, but can sometimes can be ... bad.

Hmm, I was going to keep going, but I think at some point it's a judgement call. For instance, transaction was a good one - transaction can usually be replaced with stats and be better/faster, but not quite always. Sometimes it's the only real way to get something done though (if you need all of a maxspan, an endswith clause and also a startswith clause it can be really hard to rewrite to stats...)

So maybe flag a few of the above (and similar) commands, then combine it with the stats you were thinking of ... yeah, I think this might get you at least some sort of reasonable metric by which to judge folks.

Hope this helps, and happy Splunking!
-Rich

(P.S. Note I had to strip out backticks and single quotes from the search string to make the commands work. There may be other minor gotchas to get rid of too.)

[ddrillic]

Very kind @rich7177 !

[ddrillic]

The command generates a beautiful and useful chart @rich7177.

Maybe it's also the commands which are not used such as fields.