topic Re: correlate 2 events with uid in a table in Splunk Dev

correlate 2 events with uid in a table

amir_thales — Tue, 06 Mar 2018 16:48:28 GMT

Hello,

I want information about the usb keys mounted on the system but the / var / log / messages or the /var/log/audit/audit.log do not give enough interesting information about the USB sticks.

So I want to use the data contained in the / etc / passwd and the / etc / mtab to correlate the information and thus deduce in a table the login of the current session and the name of the usb key mounted on it.

I used the add-on unix to generate the information contained in the / etc / passwd but this add-on does not propose me to generate the information of / etc / mtab, so I configured splunk for it to monitor this file and since this file is dynamic, when a line is added at the end of the file when we insert a usb key, this line is automatically generated on splunk.

Now I want to find a way to correlate message 1 with message 2.

I want to correlate the uid ie in the message 1 we see that there is the uid 500, so I will use the message 2 to deduce that 500 = local_splunk. After correlating the uids I want to make a table that will show me the uid, user, usb key name mounted.

Thank you
Amir

Re: correlate 2 events with uid in a table

elliotproebstel — Tue, 06 Mar 2018 17:19:07 GMT

If you already have the usb key name extracted into a field called usb_key, then this should work:

your current search
| eval user_id=coalesce(user_id, uid)
| stats list(user) AS user, list(usb_key) AS usb_key BY user_id

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 10:50:54 GMT

hello @elliotproebstel,

I have a table with uid and user which are correlate but i don't have the name of the USB key in the field usb_key.

amir

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 10:53:34 GMT

excuse me I did not read what was written above.

I will extract this data and test.

thank you
Amir

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 13:28:52 GMT

Would you like help writing the rex command to extract it? If so, can you show the source event and highlight which part you're extracting? I'm guessing you'd be looking at the first event for /media/Transcend, but I'm not certain. Can you also show if the usb_key value that you want to extract is currently in another field or just a part of the _raw event data?

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 13:43:36 GMT

@elliotproebstel,

yes, please.

I did this for now but I can only get the first 2 usb key not the 3:

my search | rex "(\n)(?<usb_key>/\w+/\w+\s+/\w+/\w+[-\w+])"

In the picture after, there is my event which i want use:

And what i extract with my rex:

Thank you
Amir

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 14:28:16 GMT

Does this work?

| rex "^(?<usb_key>\/\S+\s\/\S+)"

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 14:35:42 GMT

yes, it works for the first line /dev/sdb1 /media/Transcend but not for the 2 others.

it's well extracted to /dev/sdb1 /media/Transcend

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 14:36:42 GMT

but my main problem is to extract the 2 from below.

Sorry for the double post.

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 14:50:32 GMT

Got it. Does this work for you?

| rex "(^|\n)(?<usb_key>\/\S+\s\/\S+)"

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 15:00:15 GMT

it is the same problem it works but with the first line.

it extract me /dev/sdb1 /media/Transcend but not the 2 others.

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 15:11:26 GMT

Ahh, I was forgetting the max_match option, which may be biting us. Sorry for shooting in the dark a bit here, but I don't have access to a Splunk instance at the moment, so I can't test things before posting. I'm going to shotgun several options here, and you can test any/all of them:

First:

| rex max_match=0 "(?s)(?<usb_key>\/\S+\s\/\S+)"

Second:

| rex max_match=0 "(^|\n)(?<usb_key>\/\S+\s\/\S+)"

Third:

| rex max_match=0 "[\S\s](?<usb_key>\/\S+\s\/\S+)"

I'm hoping one of these works!

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 15:20:03 GMT

It's not a problem that, you take your time to help me on the contrary I'm happy to learn.

the first 2 are good, I have the 3 that are extracted.

Is it possible that you explain:

rex max_match = 0 "(^ | \ n) (? <usb_key> \ / \ S + \ s \ / \ S +)" and [\ S \ s] which is in the 2nd expression.

thank you
Amir

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 15:28:22 GMT

Great! And I'm happy to explain!

The snippet (^|\n) in the first is looking for ^, which means the start of the line OR \n, which is a newline character. I was figuring that the first line wouldn't be preceded by a newline, so we'd need ^ to grab the first one.

The [\S\s] was also a bit of a trick to look for any character at all (including a newline), because it matches on \S (any non-whitespace character) or \s (any whitespace character). But I expect it probably failed to grab the first line, since the first usb_key value had nothing at all before it. It might work if revised like this:

| rex max_match=0 "[\S\s]?(?<usb_key>\/\S+\s\/\S+)"

because that makes the [\S\s] optional (zero or one matches will pass).

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 15:33:55 GMT

ok thank you so much for your help @elliotproebstel .

I learned a lot today 🙂

Amir

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 15:41:08 GMT

My pleasure!

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 17:42:45 GMT

sorry but i want to add the time in the table but when i add it the user field becomes empty as if the correlation was no longer working.

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 18:19:36 GMT

i want to know how can i correlate uid of the message 1 with the uid in the message 2 and deduct the user and display the event when it appears with the names of the usb keys that I extracted. Because i have i problem which is, when event appear splunk only takes the usb key events alone and does not correlate the uid in the event with the uid of the message 2. is there not a function that basically allows to set up a dynamic array or when the 'event 1 appears it will directly dig into it and deduce that uid 500 = local_splunk.

Still sorry for the double post

Re: correlate 2 events with uid in a table

elliotproebstel — Wed, 07 Mar 2018 18:21:13 GMT

Sure. Given the events above, do you want to use the time from the event that contained the name of the USB key? If so, I'd do this:

your base search
| rex max_match=0 "(^|\n)(?<usb_key>\/\S+\s\/\S+)"
| eval user_id=coalesce(user_id, uid)
| eval usb_time=if(source="/etc/mtab", _time, NULL)
| stats latest(usb_time) AS _time, list(user) AS user, list(usb_key) AS usb_key BY user_id

Re: correlate 2 events with uid in a table

amir_thales — Wed, 07 Mar 2018 18:28:50 GMT

is it possible to display this by usb key with the number of times they appear each and uid and the user and the date ?

thank you
Amir