Speeding up searches that use wildcards.

[GenericSplunkUs]

Hey,

I'm trying to speed up my search and reduce its load. I'm searching through our Event Logs for Admin activity. So I've wild carded the Admin accounts with 'user="!*" so it'd pull all our admin account and non of the normal accounts. We don't have a field that specifies if it's an admin account so i don't see another way to do this. All admin accounts preface with the '!' so the wildcard search gets exactly what I want.

This is the whole search I'm using, and It's set for a 24 hour time frame in a dashboard, then puts it into a time chart.

(index=wineventlog source="WinEventLog:Security" sourcetype="XmlWinEventLog:Security" eventtype="wineventlog_security" user="!*") | fields + user | timechart count by user useother=f

The index I'm looking through is huge for a 24 hour period, about ~8 million events.

Is there a way to do this faster? Or did I just create a heavy search with nothing to do about it?

[DalJeanis]

Okay, I can't imagine that you have many tokens that start with bang, so try this instead and see what happens.

(index=wineventlog source="WinEventLog:Security" sourcetype="XmlWinEventLog:Security" eventtype="wineventlog_security" 
 "!*") | fields + user | search user="!*" | timechart count by user useother=f

In theory, this will use the bloom filters to only search the data for times when there is SOMETHING in an event that starts with !, rather than having to decode the user on every event. With luck, that's an order of magnitude faster.

[GenericSplunkUs]

This is working but it doesn't seem to have a load or speed benefit. Might even be slower. Thanks for the attempt though.

I wonder if this would be worth trying to create a summary index for? Or if that'd even be faster?

[somesoni2]

Summary indexing would definitely help.

[somesoni2]

How is the user field extracted? Just for fun see if this helps

| tstats count WHERE index=wineventlog source="WinEventLog:Security" sourcetype="XmlWinEventLog:Security"  user="!*" by _time user span=5m | timechart sum(count) by user useother=f

(index=wineventlog source="WinEventLog:Security" sourcetype="XmlWinEventLog:Security" user="!*") 
| fields _time user | timechart count by user useother=f

[GenericSplunkUs]

This isn't working, It's just telling me to increase the time. Even in 5 min of this data I'd have a couple thousand examples. I don't get any error, so somehow this just bypassed the information I'm looking for.

I haven't played with the tstat usage much, I'll give this route a try.

Thanks for the help.