topic Re: How to construct a log message containing session logs (open / close)? in Splunk Search

How to construct a log message containing session logs (open / close)?

amir_thales — Mon, 26 Jun 2017 15:42:39 GMT

Hello,

I want to build a log message that contains the logs of the same session:

login log
log of logout

And I want to use this big message log (log opening + closing) to do visualizations, basically, I want to supervise the connections and disconnections of sessions.

I think this is possible by using the PID that is associated with the created session but how to gather the log of opening and closing in a same message log and how to tell Splunk of "gathers opening and closing messages and Displays 1 message ".

If you want more explanation tell me.

Thank you
Cordially
Amir

Re: How to construct a log message containing session logs (open / close)?

horsefez — Mon, 26 Jun 2017 16:51:16 GMT

Yes, I want more explanation to be able to help you.
Please post sample data and sample querys if there are any.
Thanks!

Re: How to construct a log message containing session logs (open / close)?

somesoni2 — Mon, 26 Jun 2017 19:06:25 GMT

Grouping of those logs into one can be done using stats on pid (and other fields based on what is your final expected output) or transaction command (least recommended). As @pyro_wood mentioned, we need more details on your requirements with mock output and sample data.

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Tue, 27 Jun 2017 08:55:53 GMT

Hello pyro_wood and somesoni2,

Thanks for your quick answers

Here is my problem:

23/06/17 16:34:06,000

Jun 23 16:34:06 ccst sshd[28040]: pam_unix(sshd:session): session closed for user root

host =ccst source =/var/log/messages sourcetype =syslog

23/06/17 16:34:06,000

Jun 23 16:34:06 ccst sshd[28043]: pam_unix(sshd:session): session closed for user root

host =ccst source =/var/log/messages sourcetype = syslog

23/06/17 10:32:08,000

Jun 23 10:32:08 ccst sshd[28043]: pam_unix(sshd:session): session opened for user root by (uid=0)

host =ccst source =/var/log/messages sourcetype = syslog user =root

23/06/17 10:32:08,000

Jun 23 10:32:08 ccst sshd[28040]: pam_unix(sshd:session): session opened for user root by (uid=0)

host =ccst source = /var/log/messages sourcetype = syslog user =root

I want to group the 2 logs with the PID in bold to make a "big message" log containing the 2 and I want it to be this "big message" which is displayed and not the 2 messages alone because one can see that its Opening and closing of the same session.

Thank you
Cordially
Amir

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Tue, 27 Jun 2017 11:34:27 GMT

yes i do that but i want that when the logs arrive in the events they group the logs having the same PID together. For example, for session logs group open and close messages.

Re: How to construct a log message containing session logs (open / close)?

niketn — Tue, 27 Jun 2017 12:13:08 GMT

@amir_thales, you need to add more details, add field names/sample data for login/logout events with pid that you want to correlate.

Do you want us to help you with correlation of your logs? or creation of logs so that it can be correlated?

With the details provided so far following might work(even if it does it is not the best way).

Step 1: Correlate events

<YourBaseSearch> ("login" OR "logout") PID=*
| eval status=case(searchmatch("login"),"Login",searchmatch("logout"),"Logout")
| stats count as eventcount values(status) as status first(_time) as login_time last(_time) as last_time by PID

Step 2a: If you want to find sessions that are closed and the duration

| search  status="login" AND status="logout"
| eval duration=last_time-login_time
| eval _time=login_time
| table _time PID status eventcount duration

Or ** Step 2b: if you want to find out how long has the session been active (duration):**

| search  status="login" AND status!="logout"
| eval duration=last_time-login_time
| eval _time=login_time
| table _time PID status eventcount duration

PS: Above is based on assumption that PID is unique.

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Tue, 27 Jun 2017 13:54:58 GMT

hello niketnilay,

Thank you very much for your answer.

You understand my way but this isn't what i want because i don't want to do statistic but i want to configure splunk to regroup the log with the same PID in a big message but in the tab event.

But your proposition it is very interesting and I think I will propose it to my tutor.

Cordially
Amir
PS: sorry for my english

Re: How to construct a log message containing session logs (open / close)?

niketn — Tue, 27 Jun 2017 14:13:04 GMT

@amir_thales, what you need is on the coding side then not on Splunk side. If you want all the events with same PID to be grouped while indexing, you have to somehow write your logs only after user logs out. If the events in your logs are not even logged next to each other I am not sure how you can get them to index together as a single event.
I will leave this question unanswered for others to pitch in or for you to revert back when you have precise details your requirement.

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Tue, 27 Jun 2017 14:25:11 GMT

@niketnilay,

yes i understand, I thought it was possible to play with the logs ie to set a template so that when a message type enter this one is automatically send in a big message and this big log will be the log to display.

Thank you so much for your help because your 3 examples are very interesting and i will use them.

Cordially
Amir

Re: How to construct a log message containing session logs (open / close)?

niketn — Tue, 27 Jun 2017 21:14:32 GMT

@amir_thales, based on the sample logs that you have provided, unless you have already done so, please created field extractions for PID and session status (if required user id as well, not clear with your data/requirement whether you have/need userid).

You can try Interactive Field Extractions if you are not well versed with Regular Expression. Here is the field extraction regular expression based on your sample data...

sshd\[(?<PID>[^\]]+)\]:\spam_unix\(sshd:session\):\ssession\s(?<status>\w+)\sfor\suser\s

Give following a try if your purpose is to bring all the _raw events for the same PID together:

host =ccst source =/var/log/messages sourcetype =syslog "session"  ("closed" OR "opened")
| rex field=_raw "sshd\[(?<PID>[^\]]+)\]:\spam_unix\(sshd:session\):\ssession\s(?<status>\w+)\sfor\suser\s"
| stats list(_raw) as Events values(status) as status by PID

Events field will have Raw events stitched with PID as per your requirement.

PS: Actual Regular Expression might change based on your complete sample/test data. It is better to use Interactive Field Extraction for Splunk to generate required Regular Expression based on Sample Events.

Re: How to construct a log message containing session logs (open / close)?

horsefez — Tue, 27 Jun 2017 21:26:19 GMT

Hi Amir,

so there is a command that is known as the transaction command. It surely is nice, but should be used carefully, because it can be very demanding when it comes to search performance.

Besides the documentation I will give you an example, that could help you achieve your goal faster.
http://docs.splunk.com/Documentation/SplunkCloud/6.6.0/SearchReference/Transaction

I used the transaction command in various ways before.
I think the parameters startswithand endswith could be what you are searching for.
Maybe this example helps:
<yoursearch> | transaction pid startswith="*session opened*" endswith="*session closed*"

also this simpler version might work
<yoursearch> | transaction pid

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Wed, 28 Jun 2017 09:34:24 GMT

Hello niketnilay and pyro_wood,

Thank you for your answer.

@pyro_wood, that's exactly what i want to regroup events with the same PID and with the key word "opened" and "closed" together because i want to do a log wells.

@niketnilay, your proposition it is very interesting, it is another way to supervise its logs and i will use this method too.

Thank you very much for your help.

Cordially
Amir

Re: How to construct a log message containing session logs (open / close)?

niketn — Wed, 28 Jun 2017 12:39:17 GMT

@amir_thales, please test and let us know how it goes. Do not forget to Accept and Upvote the comments/answers that help you with your use case.

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Tue, 29 Sep 2020 14:36:20 GMT

Hello niketnilay,

Here is what I get after using the command "transaction" and the parameters "startswith" and "endswith":(This is a event)

Command:

host="ccst" session (open* OR clos*) sourcetype!="linux_audit" NOT (kernel OR "gdm-launch-environment") | transaction pid startswith="session opened" endswith="session closed"

Result:

 28/06/17 11:10:13,000  
 Jun 28 11:10:13 ccst sshd[4444]: pam_unix(sshd:session): session opened for user splunker by (uid=0)
 Jun 28 11:11:05 ccst sshd[4444]: pam_unix(sshd:session): session closed for user splunker

  host =ccst      source =/var/log/messages      sourcetype =syslog      user =splunker

I have the 2 session logs that are grouped together with the same PID because it is the same session.

Thank you
Cordially
Amir

Re: How to construct a log message containing session logs (open / close)?

niketn — Tue, 29 Sep 2020 14:40:28 GMT

@amir_thales, please accept @pyro_woods answer if you want to stick to transaction command. However, transaction is not the correct use case, and if you can achieve the same result with stats command, you should actually use that instead because performance will be way better with stats which uses map-reduce.

I am glad we were able to assist with your use case.

Re: How to construct a log message containing session logs (open / close)?

amir_thales — Thu, 29 Jun 2017 08:08:11 GMT

Hello niketnilay,

It is true when i want to display something in the tab event, it is very low whereas with statistic it comes more quickly.

Cordially
Amir

PS: i cannot accept the @pyro_wood's message because it is a comment but i upvote his comment.

Re: How to construct a log message containing session logs (open / close)?

aaraneta_splunk — Tue, 29 Sep 2020 14:41:14 GMT

Hi @amir_thales - I just converted pyro_wood's comment to an answer so you can "Accept" their answer and close your question. Thank you.