All Apps and Add-ons
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question
Solved! Jump to solution

regex - bash history

szaboszilard
Path Finder
‎02-07-2014 12:41 AM

Hi geeks,

I have a file included some different lines about users command history. Example:

2014-02-07T08:37:44.943764+01:00 hostname2 [audit / as user2/15313 on /192.168.199.211:41276->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:37:44.923455+01:00 hostname2 [audit / as user2/15313 on /192.168.199.211:41276->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:37:43.810067+01:00 hostname2 [audit / as user2/15276 on /192.168.199.211:41275->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:37:43.808603+01:00 hostname2 [audit / as user2/15276 on /192.168.199.211:41275->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:35:47.141732+01:00 hostname2 [audit / as user2/14955 on /192.168.199.211:41061->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:35:47.123633+01:00 hostname2 [audit / as user2/14955 on /192.168.199.211:41061->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:30:51.569551+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: su -
2014-02-07T08:30:45.975700+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: su -
2014-02-07T08:30:38.609552+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: df -h
2014-02-07T08:30:36.730315+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: ls -ali
2014-02-06T12:04:54.717480+01:00 hostname3 [audit / as user1/6898 on /192.168.200.201:9606->192.168.240.243:22] /home/user1: PROMPT_COMMAND='pwd>&7;kill -STOP $$'
2014-02-07T08:33:18.764505+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22]: #=== session closed ===
2014-02-07T08:33:18.042308+01:00 hostname3 [audit user1/23338 as root/23676 on pts/1/]: #=== session closed ===
2014-02-07T08:33:17.107657+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/]: #=== session closed ===
2014-02-07T08:33:09.716782+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: cat /var/log/userlog.info 
2014-02-07T08:33:05.195923+01:00 hostname2 [audit / as user2/14653 on /192.168.22.201:35868->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:33:02.553277+01:00 hostname2 [audit / as user2/14616 on /192.168.22.201:35865->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:33:02.551895+01:00 hostname2 [audit / as user2/14616 on /192.168.22.201:35865->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:33:00.749622+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: teszt
2014-02-07T08:32:58.262602+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: id
2014-02-07T08:32:57.384724+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: ls -ali
2014-02-07T08:32:55.758173+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: ls
2014-02-07T08:32:53.056071+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/]: #=== session opened ===
2014-02-07T08:32:52.941806+01:00 hostname3 [audit user1/23338 as root/23676 on pts/1/] /root: screen 
2014-02-07T08:32:42.845085+01:00 hostname2 [audit / as user2/14578 on /192.168.20.201:59883->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:42.759075+01:00 hostname2 [audit / as user2/14578 on /192.168.20.201:59883->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:42.670319+01:00 hostname2 [audit / as user2/14541 on /192.168.21.201:59612->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:42.505636+01:00 hostname2 [audit / as user2/14541 on /192.168.21.201:59612->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:41.995335+01:00 hostname2 [audit / as user2/14504 on /192.168.21.201:59611->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:41.993945+01:00 hostname2 [audit / as user2/14504 on /192.168.21.201:59611->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:41.934074+01:00 hostname2 [audit / as user2/14470 on /192.168.20.201:59882->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:41.932644+01:00 hostname2 [audit / as user2/14470 on /192.168.20.201:59882->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:37.407663+01:00 hostname3 [audit user1/23338 as root/23676 on pts/1/] /root: cat /var/log/userlog.info

I'm writing an regex, but i have a problem.

\[audit\s(?<src_user>.*)\sas\s(?<dst_user>.*)\son(?<console>\s|\spts/\d)\/(?<src_ip>.*)\:(?<src_port>\d*)-\>(?<dst_ip>.*)\:(?<dst_port>\d*)\](\s|\:\s)(\#\=\=\=\ssession\sclosed\s\=\=\=|\#\=\=\=\ssession\sopened\s=\=\=|(?<src_path>.*)\:(?<command>.*))

If i use switch user command, the process dose not generate any src/dst ip/port in logfile.
My regex don't match on any line. Any idea how can i resolve this?

not matching lines:

2014-02-07T08:33:00.749622+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: teszt
2014-02-07T08:32:58.262602+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: id
2014-02-07T08:32:57.384724+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: ls -ali
2014-02-07T08:32:55.758173+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: ls

Regards Szilard

Tags (3)
0 Karma
Reply
1 Solution
Solved! Jump to solution
Solution

kristian_kolb
Ultra Champion
‎02-07-2014 02:26 AM

Instead of writing a rather complex regex, it would probably be easier to write a different one for each pattern, e.g.

props.conf

EXTRACT-foo = \[audit\s(?<src_user>\S+)\sas\s(?<dst_user>\S+)\son
EXTRACT-bill = \son\s(?<console>pts[^\]]+)\]
EXTRACT-bob = \son\s(?<src_ip>[\d.]+):(?<src_port>\d+)-\>(?<dst_ip>[\d.]+):(?<dst_port>\d+)\]
EXTRACT-jim = \]\s(?<path>/\S+)\s(?<command>.*)
EXTRACT-joe = \]:\s\#===\ssession\s(?<session_status>\w+)
EXTRACT-hoss = \s\on\s(?<origin>\S+)\]
EXTRACT-hank = \s\S+\][^:]*:\s(?<action>.*)

This means that not all fields will be present in all events, e.g. src_port etc, and that some extractions will overlap.

The extraction named foo will handle the things that are common to all events, whereas bill and bob will deal with the alternate versions of pts* and ip/port pairs.
jim and joe deal with the different actions (sessions and commands executed). Finally hank & hoss extracts jim/joe and bill/bob information, regardless of format for easier reporting.

Consider this as examples of an approach, rather than The One Way To Do It.

/K

View solution in original post

Reply
Solved! Jump to solution
Solution

kristian_kolb
Ultra Champion
‎02-07-2014 02:26 AM

Instead of writing a rather complex regex, it would probably be easier to write a different one for each pattern, e.g.

props.conf

EXTRACT-foo = \[audit\s(?<src_user>\S+)\sas\s(?<dst_user>\S+)\son
EXTRACT-bill = \son\s(?<console>pts[^\]]+)\]
EXTRACT-bob = \son\s(?<src_ip>[\d.]+):(?<src_port>\d+)-\>(?<dst_ip>[\d.]+):(?<dst_port>\d+)\]
EXTRACT-jim = \]\s(?<path>/\S+)\s(?<command>.*)
EXTRACT-joe = \]:\s\#===\ssession\s(?<session_status>\w+)
EXTRACT-hoss = \s\on\s(?<origin>\S+)\]
EXTRACT-hank = \s\S+\][^:]*:\s(?<action>.*)

This means that not all fields will be present in all events, e.g. src_port etc, and that some extractions will overlap.

The extraction named foo will handle the things that are common to all events, whereas bill and bob will deal with the alternate versions of pts* and ip/port pairs.
jim and joe deal with the different actions (sessions and commands executed). Finally hank & hoss extracts jim/joe and bill/bob information, regardless of format for easier reporting.

Consider this as examples of an approach, rather than The One Way To Do It.

/K

Reply
Solved! Jump to solution

szaboszilard
Path Finder
‎02-07-2014 03:03 AM

Today I Learned Something New 🙂
great, thank you very much

0 Karma
Reply
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...