topic Re: Correlate three events with differing properties in Splunk Search

Correlate three events with differing properties

AndreasBalster — Fri, 02 May 2014 11:37:45 GMT

I need to correlate three events of different type which have 1 single property in common, respectively:

<TS> type_name=AUDIT_PATH callid=123 exe=/etc/sudoers.work
<TS> type_name=AUDIT_SYSCALL callid=123 session=456
<TS> type_name=AUDIT_USER_START session=456 acct=root

My tries so far have been unsuccessful and I'm running out of ideas, how can I do this?

P.S. I'm trying to not use the transaction command

Re: Correlate three events with differing properties

richgalloway — Fri, 02 May 2014 12:31:59 GMT

Have you tried something like this?

source=foo type_name=AUDIT_PATH | join callid [search type_name=AUDIT_SYSCALL] | join session [search type_name=AUDIT_USER_START]

Re: Correlate three events with differing properties

AndreasBalster — Fri, 02 May 2014 14:48:40 GMT

This produces something, but I cannot verify its correctness or completeness. Judging by the warning message Splunk gives me (Subsearch exceeded 50000 events, using only first 50000) I think it will be incomplete. Thank you for your reply nonetheless!

Re: Correlate three events with differing properties

MuS — Mon, 05 May 2014 06:52:47 GMT

Hi AndreasBalster,

you can use streamstats for this. Given this log file:

<TS> type_name=AUDIT_PATH callid=123 exe=/etc/sudoers.work
<TS> type_name=AUDIT_SYSCALL callid=123 session=456
<TS> type_name=AUDIT_USER_START session=456 acct=root
<TS> type_name=AUDIT_PATH callid=124 exe=/etc/foo.work
<TS> type_name=AUDIT_SYSCALL callid=124 session=457
<TS> type_name=AUDIT_USER_START session=457 acct=root
<TS> type_name=AUDIT_PATH callid=125 exe=/etc/boo.work
<TS> type_name=AUDIT_SYSCALL callid=125 session=458
<TS> type_name=AUDIT_USER_START session=458 acct=root
<TS> type_name=AUDIT_PATH callid=126 exe=/etc/sudoers.work
<TS> type_name=AUDIT_SYSCALL callid=126 session=459
<TS> type_name=AUDIT_USER_START session=459 acct=root2
<TS> type_name=AUDIT_PATH callid=127 exe=/etc/foo.work
<TS> type_name=AUDIT_SYSCALL callid=127 session=460
<TS> type_name=AUDIT_USER_START session=460 acct=root2
<TS> type_name=AUDIT_PATH callid=128 exe=/etc/bla.work
<TS> type_name=AUDIT_SYSCALL callid=128 session=461
<TS> type_name=AUDIT_USER_START session=461 acct=root2
<TS> type_name=AUDIT_PATH callid=129 exe=/etc/sudoers.work
<TS> type_name=AUDIT_SYSCALL callid=129 session=462
<TS> type_name=AUDIT_USER_START session=462 acct=root2

I was able to use this search and got back a nice table of callid, session, exe and acct

source=/var/tmp/myfoo type_name=AUDIT_PATH OR type_name=AUDIT_USER_START OR type_name=AUDIT_SYSCALL | streamstats current=f last(callid) AS last_callid last(session) AS last_session last(acct) AS last_acct  | where callid=last_callid | rename last_session AS session last_acct AS acct | table callid session acct exe

Since you got millions of events I cannot tell if this will perform at a good rate or if it will work at all, since I used the above test file and not your real world data.

hope this helps ...

cheers, MuS

Re: Correlate three events with differing properties

AndreasBalster — Tue, 06 May 2014 07:17:17 GMT

Sadly, this approach seems to be dependent on the order of events.. My data gets correlated in a non-matching fashion (it puts exes together with users that didn't call those). But many thanks for your reply!

Re: Correlate three events with differing properties

somesoni2 — Tue, 06 May 2014 13:21:34 GMT

What is your expected output out of this? More information can help get better answers.