Hi,
We have Linux Auditd data coming into Splunk with sourcetype=linux:audit
. In Auditd logs, Record Types define events based on what information is being presented, e.g. a type of "CWD" is triggered to record current working directory, type "SYSCALL" records a system call to kernel etc. Full list here in case anyone's interested. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security_guide/sec-aud...
To find out when any user runs a potentially risky command e.g. rm -rf XYZ or chmod 777 XYZ, I use the following search which joins three types(CWD, EXECVE and SYSCALL) based on msg
field which holds the id unique across all types for a specific Event.
sourcetype=linux:audit type=CWD
| fields + msg, cwd
| JOIN msg [ search sourcetype=linux:audit NOT auid=4294967295 type=SYSCALL comm=chmod OR comm=rm OR comm=chown
| fields + _time, msg, auid]
| JOIN msg [ search sourcetype=linux:audit NOT auid=4294967295 type=EXECVE (a0=chmod (a1=-R a2=777 OR a2=755) OR (a1=777)) OR (a0=rm a2=-r OR a2=-rf) OR (a0=chown) | fields + _time, msg, host, a0, a1, a2, a3] | table _time, host, msg, auid, a0, a1, a2, a3, cwd
Search works fine and gets the results but it takes exceptionally long to execute and almost every time I see the following under job summary.
info : [subsearch]: Search Processor: Subsearch produced 50000 results, truncating to maxout 50000.
warn : The limit has been reached for log messages in info.csv. 20 messages have not been written to info.csv. Please refer to search.log for these messages or limits.conf to configure this limit.
Is the search missing out on some data? Is there a way to make search more efficient and ensure it searches every event in the given time range?
Any helps is appreciated..
~ Abhi
Hi abhijittikekar,
Try to run below query.
sourcetype=linux:audit type=CWD
| table msg, cwd
| map
[ search sourcetype=linux:audit NOT auid=4294967295 type=SYSCALL comm=chmod OR comm=rm OR comm=chown msg=$msg$
|eval cwd=$cwd$
| table _time, msg, auid,cwd]
|map
[ search sourcetype=linux:audit NOT auid=4294967295 msg=$msg$ type=EXECVE (a0=chmod (a1=-R a2=777 OR a2=755) OR (a1=777)) OR (a0=rm a2=-r OR a2=-rf) OR (a0=chown)
| eval auid=$auid$ , cwd=$cwd$
| table + _time, msg, host, a0, a1, a2, a3,auid,cwd]]
| table _time, host, msg, auid, a0, a1, a2, a3, cwd
Thanks,
Bhavik
Like this:
(index=YouShouldAlwaysSpecifyIndexValues AND sourcetype=linux:audit) AND
((type="CWD")
OR
(NOT auid="4294967295" AND type="SYSCALL" AND (comm="chmod" OR comm="rm" OR comm="chown"))
OR
(NOT auid="4294967295" AND type="EXECVE" AND (a0="chmod" AND (a1="-R" AND (a2="777" OR a2="755")) OR (a1="777")) OR (a0="rm" AND a2="-r*") OR (a0="chown")))
| fields _time msg, cwd, auid, host, a0, a1, a2, a3
| stats values(*) AS * BY msg
index=?? sourcetype=linux:audit type=CWD OR ( index= ?? sourcetype=linux:audit NOT auid=4294967295 type=SYSCALL comm=chmod OR comm=rm OR comm=chown or (INDEX=?? sourcetype=linux:audit NOT auid=4294967295 type=EXECVE (a0=chmod (a1=-R a2=777 OR a2=755) OR (a1=777)) OR (a0=rm a2=-r OR a2=-rf) OR (a0=chown) )
| fields + _time, msg, auid, cwd,host, a0, a1, a2, a3]
| stats values(*) as * by msg
You can use stats commands to accomplish the same thing:
| stats values(field1) as field1, values(field2) as field2 by msg
Watch this video, apply what you learned and your search will be significantly faster
https://conf.splunk.com/files/2018/recordings/master-joining-datasets-without-fn1784.mp4
Thanks for the reference. There is tons of information there.. going through the content. i'll re post updated search after making changes.
~ Abhi