topic Re: Query with Transaction Help/Optimize in Splunk Search

Query with Transaction Help/Optimize

zafunt — Thu, 13 Mar 2014 14:20:28 GMT

My search is ...

sourcetype=linux_audit (type="SYSCALL" OR type="PATH") | transaction host lin_audit_event maxevents=3 maxspan=24h keepevicted=true | search success="no" (name="/etc/*" OR name="/bin/*" OR name="/usr/sbin/*")

This query is taking a very long time to run, are there any optimizations that I can make?
If a large time range is chosen, the query takes 15+ minutes, and gives me the error that "No matching fields exist". However, if I add host="whateverhost" to the search, I see results. Am I missing some parameters?

Re: Query with Transaction Help/Optimize

somesoni2 — Thu, 13 Mar 2014 18:48:07 GMT

how about moving your second search filters (succes="no"...) to base search filter? This should make the query little faster.

Re: Query with Transaction Help/Optimize

the_wolverine — Thu, 13 Mar 2014 19:25:20 GMT

The maxspan makes a huge difference. If you can set a lower maxspan, like 1h or even less, it will be significantly faster.

Re: Query with Transaction Help/Optimize

zafunt — Fri, 14 Mar 2014 10:55:18 GMT

I've tried the maxspan=5s (even went to 24h). However, my audits look like this in splunk...
3/11/14 8:45:58.171 AM, type=SYSCALL, exe="/usr/bin/rm", uid=1001
3/11/14 8:01:00.000 AM, type=PATH, item=1, name="/etc/shadow"
3/11/14 8:01:00.000 AM, type=PATH, item=0, name="/etc"

These events are created in the audit.log almost instantly, but it looks like the _time is off for the events that are type=PATH.

Re: Query with Transaction Help/Optimize

zafunt — Fri, 14 Mar 2014 14:50:34 GMT

Thanks for the suggestion.

now... Please forgive the syntax, I can't copy-paste.

sourcetype=linux_audit (type="SYSCALL" success="no") OR (type="PATH" (name="/etc/" OR name="/bin/" OR name="/usr/sbin*")) | transaction maxevents=3 maxspan=24h keepevicted=true

This seems to be more optimized. However, I am getting the same results. Am, I hitting some kind of limit?

Re: Query with Transaction Help/Optimize

zafunt — Tue, 29 Sep 2020 10:03:55 GMT

I hate to rehash an old thread, but I was able to speed up the query significantly by following the comments from somesoni2 and the_wolverine above. In addition, I was able to create a props.conf file to resolve the timestamp issue mentioned in my last comment.

[source::.../var/log/audit/audit.log(.\d+)?]
TIME_PREFIX = audit(
TIME_FORMAT = %s.%3N

Of course, this would probably be resolved by using rlog.sh in Splunk_TA_Nix app, but it blows up with some older Suse boxes.