topic Re: How to pick overall message in Splunk Search

How to pick overall message

vikas_gopal — Wed, 07 Jan 2015 13:20:22 GMT

Hi Experts,
I have syslog file and I want to generate a table from this log file .This file contains log like

2014-12-26 16:22:42 Local7.Debug    172.28.70.132   CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-
Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network.

I want to pick complete "msg" from this log but in the table when I pick msg it shows only one word "Network". I want to show complete msg field like "Network: A user or computer logged on to this computer from the network."
Please help on this .

Re: How to pick overall message

gabetheISguy — Wed, 07 Jan 2015 13:28:17 GMT

Splunk auto extracts fields when fields=value criteria is met. Since the event has "msg=Network: " it auto extracts the msg field as Network

To overcome this I would use a regular expression with named capturing groups to extract specific fields as per the requirement

For e.g: to Capture the message using a named capturing group i would use this. Having said that using .* is not always the best method as it captures everything else after the msg actually ends. The field named as Message will capture the messages. LMK if this works

(?i)msg=(?P<Message>.*)

Re: How to pick overall message

MuS — Wed, 07 Jan 2015 13:28:42 GMT

Hi vikas_gopal,

how about this:

your base search here | rex "\smsg\=(?<myMsg>.+?)\." | table myMsg

assumption there is no new line, line break after msg=

hope this helps ...

cheers, MuS

Re: How to pick overall message

vikas_gopal — Wed, 07 Jan 2015 13:40:44 GMT

this is my first time that I am using REX . This is what I have done , but with this message is blank.
sourcetype="CEF" | table cat,msg | rex field=msg (?i)msg=(?P.*)

Re: How to pick overall message

vikas_gopal — Wed, 07 Jan 2015 13:41:25 GMT

This is what I have done , but with this message is blank.
sourcetype="CEF" | table cat,msg | rex field=msg (?i)msg=(?P.*)
even I tried
sourcetype="CEF" | table cat,msg | rex field=msg "\smsg=(?.+?)."

Re: How to pick overall message

MuS — Wed, 07 Jan 2015 13:43:07 GMT

this is because your field msg only contains Network use the rex like this:

sourcetype="CEF" | rex  "(?i)msg=(?P<msg>.*)" | table cat,msg

Re: How to pick overall message

MuS — Wed, 07 Jan 2015 13:45:30 GMT

try this :

sourcetype="CEF" | rex "\smsg\=(?<msg>.+?)\." | table cat, msg

Re: How to pick overall message

vikas_gopal — Wed, 07 Jan 2015 13:59:48 GMT

yup it works , the only problem is it is picking everything after msg field . Can we do something like it picks line between "msg" and "categorySignificance" fields . I observed that message ends just before "categorySignificance" field.
Here is the overall log

2014-12-26 16:22:42    Local7.Debug    172.28.70.132    CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network. categorySignificance=/Informational categoryBehavior=/Access/Stop categoryDeviceGroup=/Operating System catdt=Operating System categoryOutcome=/Success categoryObject=/Host/Operating System art=1419639752421 cat=Security deviceSeverity=Audit_success rt=1419639736000 dhost=WIN-789Z3VMFC02 dst=172.28.70.132 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 172.16.0.0-172.31.255.255 dntdom=WIN-789Z3VMFC02 duser=Administrator duid=0x1d11f9 cs2=Logon/Logoff:Logoff cn1=3 c6a4=fe80:0:0:0:8045:2eed:5e73:a468 cs1Label=Accesses cs2Label=EventlogCategory cs4Label=Reason or Error Code cs5Label=Authentication Package Name cn1Label=LogonType cn2Label=CrashOnAuditFail cn3Label=Count c6a4Label=Agent IPv6 Address ahost=WIN-789Z3VMFC02 agt=1

Re: How to pick overall message

MuS — Wed, 07 Jan 2015 14:03:22 GMT

try my below provided regex, this will stop at the .

Re: How to pick overall message

vikas_gopal — Wed, 07 Jan 2015 14:07:23 GMT

I tried above command only it is still picking everything after "msg" ,like till the last word of the overall log

Re: How to pick overall message

MuS — Wed, 07 Jan 2015 14:11:01 GMT

Try this regex here, see above ↑

Re: How to pick overall message

vikas_gopal — Wed, 07 Jan 2015 14:13:23 GMT

2014-12-26 16:22:42    Local7.Debug    172.28.70.132    CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network. categorySignificance=/Informational categoryBehavior=/Access/Stop categoryDeviceGroup=/Operating System catdt=Operating System categoryOutcome=/Success categoryObject=/Host/Operating System art=1419639752421 cat=Security deviceSeverity=Audit_success rt=1419639736000 dhost=WIN-789Z3VMFC02 dst=172.28.70.132 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 172.16.0.0-172.31.255.255 dntdom=WIN-789Z3VMFC02 duser=Administrator duid=0x1d11f9 cs2=Logon/Logoff:Logoff cn1=3 c6a4=fe80:0:0:0:8045:2eed:5e73:a468 cs1Label=Accesses cs2Label=EventlogCategory cs4Label=Reason or Error Code cs5Label=Authentication Package Name cn1Label=LogonType cn2Label=CrashOnAuditFail cn3Label=Count c6a4Label=Agent IPv6 Address ahost=WIN-789Z3VMFC02 agt=1