How to write a search based on Unique Ids considering multi-line to a single line?

[Thulasinathan_M]

Hi Splunk Experts,

I've logs where users activites are tracked based on a unique identifiers, I want to display the logs where username is logged, along with couple of lines above and below the username.

But the complication here is for the same unqiue identifiers at times username will be logged in more than one line so each logged lines are considered as a new activity, but I want to considet them as a single activity and print only the first match with 4 to 5 lines. 

Any idea/ suggestion would be much appreciated. Thanks in advance.

[ITWhisperer]

If I understand correctly,  you want to find the first username event for each unique id, and then just keep events for that uid which are close to it on both sides? Try something like this:

| streamstats count(eval(event=="Username event")) as usernameinstance count as uidevents by uid
| eval usernameinstance=if(event=="Username event" and usernameinstance==1,usernameinstance,null())
| eventstats values(eval(if(usernameinstance == 1, uidevents, null()))) as firstusernameevent by uid
| where uidevents > firstusernameevent - 3 and uidevents < firstusernameevent + 3

[Thulasinathan_M]

Hi @ITWhisperer

Thanks for your prompt response. Let me try to explain with an example.

[UID:tdvpoibdg65dr1] Audit page has been launched
[UID:tdvpoibdg65dr1] User Mike started accessing the audit page.
[UID:tdvpoibdg65dr1] The DailyTracker audit_event_status has been altered from 'Pending' to 'In Progress'
[UID:tdvpoibdg65dr1] The name of the auditer is changed
[UID:tdvpoibdg65dr1] The DailyTracker page has been updated successfully. 
[UID:tdvpoibdg65dr1] The DailyTracker audit_event_status has been altered from 'In Progress' to 'Done'
[UID:tdvpoibdg65dr1] The DailyTracker page has been updated successfully. 
[UID:tdvpoibdg65dr1] The DailyTracker page closed.

From the above logs, I'm trying to search for 'status' and I've managed to get below 2 lines and not the above 2 lines. My issue here is 'audit_event_status' logged two times but they're asscociated to same UID(activity).

query used:
index=audit ".*status"
| regex event="UID((.*(\n|\r\n|\r)){0,2}).*status.*"


My Expectation:
[UID:tdvpoibdg65dr1] Audit page has been launched
[UID:tdvpoibdg65dr1] User Mike started accessing the audit page.
[UID:tdvpoibdg65dr1] The DailyTracker audit_event_status has been altered from 'Pending' to 'In Progress'
[UID:tdvpoibdg65dr1] The name of the auditer is changed
[UID:tdvpoibdg65dr1] The DailyTracker page has been updated successfully. 

The result I got:
Event 1:
[UID:tdvpoibdg65dr1] The DailyTracker audit_event_status has been altered from 'Pending' to 'In Progress'
[UID:tdvpoibdg65dr1] The name of the auditer is changed
[UID:tdvpoibdg65dr1] The DailyTracker page has been updated successfully. 

Event 2:
[UID:tdvpoibdg65dr1] The DailyTracker audit_event_status has been altered from 'In Progress' to 'Done'
[UID:tdvpoibdg65dr1] The DailyTracker page has been updated successfully. 
[UID:tdvpoibdg65dr1] The DailyTracker page closed.

[ITWhisperer]

Are each of the lines beginning with the UID separate events in Splunk or a single multi-line event holding all of the lines?

[Thulasinathan_M]

They're separate events. Random UID gets generated once the user logged in and it'll maintained until log off.

User each login will have a UniqueIDs(UID).

[ITWhisperer]

| rex "UID:(?<uid>[^\]]+)"
| streamstats count(eval(match(_raw,"_status"))) as usernameinstance count as uidevents by uid
| eval usernameinstance=if(match(_raw,"_status") and usernameinstance==1,usernameinstance,null())
| eventstats values(eval(if(usernameinstance == 1, uidevents, null()))) as firstusernameevent by uid
| where uidevents > firstusernameevent - 3 and uidevents < firstusernameevent + 3

[Thulasinathan_M]

Hi @ITWhisperer

Thanks for the prompt response and apologies for the delay to get back to you, I can't assure status always prefix with '_' at time it might be prefixed & suffixed with Chars & Special chars.  

match(_raw,"_status")

 I'm new to splunk world, I didn't understand this part, shall I request you kindly to explain below piece

| where uidevents > firstusernameevent - 3 and uidevents < firstusernameevent + 3

[ITWhisperer]

The match string just has to be a string that occurs first in the events for the unique id around which you want the other events. From your example "_status" fit the bill. You should change this to something from your real events which identifies the middle event.

The where command keeps events which are -/+ 2 of the middle event previously identified. Try it out, adding one line at a time to see what it does.