Is there a feature in the splunk windows add-on to add data value called security_group_id from windows security event log as a individual splunk fields ?

[Hemnaath]

Hi All, Currently in splunk, many of the windows log have multiple account ,domains,names etc and they all parse the same way.

Example

index=windows sourcetype="WinEventLog:Security" EventCode=4728 

LogName=Security
 SourceName=Microsoft Windows security auditing.
 EventCode=4728
 EventType=0
 Type=Information
 ComputerName=VMDC01.XXXXXXs.com
 TaskCategory=Security Group Management
 OpCode=Info
 RecordNumber=666661820
 Keywords=Audit Success
 Message=A member was added to a security-enabled global group.

 Subject:
     Security ID:        XXXXX\TEST01
     Account Name:        TEST01
     Account Domain:        TEST
     Logon ID:        0xB7D860D0

 Member:
     Security ID:        XXXXXX\HXXX005
     Account Name:        CN=LXXXX\, Lucy,OU=Disabled Users,DC=XXXXXXs,DC=com

 Group:
     Security ID:        XXXXXX\OutlookCachedModeUsers
     Group Name:        OutlookCachedModeUsers
     Group Domain:        XXXXXX

 Additional Information:
     Privileges:

The resulting logs have a "Group", "Subject" and "Member" that all have a "Security ID" field in them. These need to be parsed individually as Group_Security_ID, Subject_Security_ID, and Member_Security_ID. This should be done for each sub-field.

Kindly guide me on this.

[dstaulcu]

I would not use xml rendered events unless (1) there is data you need in the EventData subsection of XML that is not also in the Message field or (2) your Splunk users are competent with t-stats based event data exploration. XML field extraction is incredibly expensive at search time for dense data sets like the windows security logs.

Here is a query I use to differentiate between the subject and member security ids. In cases where differentiation matters I do not rely on Splunk_TA_Windows auto-extractions. I used custom field extractions instead.

sourcetype=WinEventLog:Security EventCode=4728 
| rex field=Message "(?<summary>.*)" 
| rex field=Message "Subject:\s+Security ID:\s+(?<Subject_Security_ID>.*)\s+Account Name:\s+(?<Subject_Account_Name>.*)\s+Account Domain:\s+(?<Subject_Account_Domain>.*)\s+Logon ID:\s+(?<Subject_Logon_ID>.*)"
| rex field=Message "Member:\s+Security ID:\s+(?<Member_Security_ID>.*)\s+Account Name:\s+(?<Member_Account_Account_Name>.*)"
| rex field=Message "Group:\s+Security ID:\s+(?<Group_Security_ID>.*)\s+Group Name:\s+(?<Group_Group_Name>.*)\s+Group Domain:\s+(?<Group_Group_Domain>.*)"
| table _time host sourcetype EventCode summary, Subject_Security_ID, Subject_Account_Name, Subject_Account_Domain, Subject_Logon_ID, Member_Security_ID Member_Account_Account_Name Group_Security_ID Group_Group_Name Group_Group_Domain

[xpac]

Thanks for pointing this out, the XML field extraction/performance issue is something I didn't have in mind, and yeah, they're pretty dense, so that would most likely be a problem. 🙂
I'll steal your regexes, because I thought of doing those, too. 🙂

[Hemnaath]

xpac, I have gone through the Props.conf document but not sure how to begin. So Could you please guide me from where I need to start the stanza and how to create the stanza based on the requirement.

link: https://docs.splunk.com/Documentation/Splunk/7.1.0/Admin/Propsconf#Field_extraction_configuration

thanks in advance.

[Hemnaath]

thanks dstaulcu for your effort on this. Actually my client wants to get this filed value auto extracted instead executing the query by the end users. So could you please guide me on how to create a props & transforms stanza in order to get above same results automatically, with out executing the customized query.

thanks in advance.

[xpac]

You need a props entry for the sourcetype WinEventLog:Security, that includes a FEW REPORT- settings in it. Make sure these come after those already shipped with Splunk, so that the Message field actually exists. You can check for that using splunk btool props list WinEventLog:Security, and see which REPORT- or EXTRACT- creates tue Message field.

After that, create some transforms stanzas according to what you mentioned in props.conf, and simply put in the regex from above, SOURCE_KEY is Message, and you should be pretty good to go.

There is a bunch of documentation on all of those topics available, so I think you should make it from here 😉

[Hemnaath]

thanks xpac, for your time on this. I could see the below stanza details in the props.conf, but not sure where and what exactly needs to be created.

Props.conf Details: 
###### Windows Security Event Log ######
[source::*:Security]

## action
LOOKUP-action_for_windows1_security = windows_action_lookup Type OUTPUTNEW action
LOOKUP-action_for_windows2_security =  windows_action_lookup Type as Keywords OUTPUTNEW action

## privilege
REPORT-0vendor_privilege_for_windows_security = vendor_privilege_sv_for_windows_security,vendor_privilege_mv_for_windows_security
REPORT-privilege_id_for_windows_security = privilege_id_for_windows_security
LOOKUP-privilege_for_windows_security = windows_privilege_lookup privilege_id OUTPUT privilege

FIELDALIAS-src_port_for_windows_security = Source_Port as src_port
REPORT-Token_Elevation_Type_id_for_windows_security = Token_Elevation_Type_id_for_windows_security
LOOKUP-vendor_info_for_windows_security = windows_vendor_info_lookup sourcetype OUTPUT vendor,product

## Set the app field to "win:remote" or "win:local" based on EventCode, Source_Network_Address, Target_Server_Name or Logon_Type
LOOKUP-app0_for_windows_security = windows_app_lookup EventCode OUTPUTNEW app
LOOKUP-app1_for_windows_security = windows_app_lookup Source_Network_Address OUTPUTNEW app
LOOKUP-app2_for_windows_security = windows_app_lookup Target_Server_Name OUTPUTNEW app
LOOKUP-app3_for_windows_security = windows_app_lookup Logon_Type OUTPUTNEW app
LOOKUP-app4_for_windows_security = windows_app_lookup sourcetype OUTPUTNEW app

## Set the following fields based on order of operations
REPORT-dest_for_windows_security = Target_Server_Name_as_dest,ComputerName_as_dest
REPORT-dest_nt_domain_for_windows_security = Target_Domain_as_dest_nt_domain,Primary_Domain_as_dest_nt_domain,Account_Domain_as_dest_nt_domain,New_Domain_as_dest_nt_domain,Domain_as_dest_nt_domain
REPORT-dest_nt_host_for_windows_security = Target_Server_Name_as_dest_nt_host,ComputerName_as_dest_nt_host
REPORT-src_for_windows_security = Workstation_Name_as_src,Caller_Machine_Name_as_src,Client_Machine_Name_as_src,Source_Network_Address_as_src,Client_Address_as_src,ComputerName_as_src
REPORT-src_ip_for_windows_security = Source_Network_Address_as_src_ip,Client_Address_as_src_ip
REPORT-src_nt_domain_for_windows_security = Caller_Domain_as_src_nt_domain,Client_Domain_as_src_nt_domain,Account_Domain_as_src_nt_domain
REPORT-src_nt_host_for_windows_security = Workstation_Name_as_src_nt_host,Caller_Machine_Name_as_src_nt_host,Client_Machine_Name_as_src_nt_host
REPORT-src_user_for_windows_security = Caller_User_Name_as_src_user,Client_User_Name_as_src_user,Account_Name_as_src_user,User_Name_as_src_user
REPORT-user_for_windows_security = Target_User_Name_as_user,Primary_User_Name_as_user,Target_Account_Name_as_user,New_Account_Name_as_user,Account_Name_as_user,User_Name_as_user,User_as_user 

Yes, I have gone through the Props.conf document but not sure how to start. So Could you please guide me from where I need to start the stanza and how to create the stanza based on the requirement. It will be great if you can provide me an example so that I can try myself first.

thanks in advance.

[dstaulcu]

If yall use GitHub, I can add you guys as contributors to a GitHub repo (fork of Splunk_TA_Windows) where we can all contribute towards a workable integration...

[dstaulcu]

can't think of a way to do that off hand but I will challenge myself to figure it out.

[dstaulcu]

Challenge completed. https://github.com/dstaulcu/Splunk_SavedSearches

[Hemnaath]

Hi dstaulcu, thanks and sincerely appreciate for your time and effort. great work !!!

hey do I need to upload all the file which are present in the github to the Splunk_TA_windows app folder to achieve the results. Kindly guide me on this.

[dstaulcu]

you could take the sections of props and transforms from GitHub and integrate them within your copy of Splunk_TA_windows or create your own app, drop the files in it's default folder, and then make sure the objects are accessible to all and global. I would recommend challenging yourself take it the rest of the way!

[Hemnaath]

thanks dstaulcu, I am trying to seek help from xpac on this.

[xpac]

Hey, if you configure the input to render the logs as XML, the fields will have proper unique names, instead of what you experienced.
You could also do this manually for all of them, but it's some work and not very flexible.

[Hemnaath]

xpac, thanks for your response, but i am not sure whether you had understood the requirement correctly or not. we need below details to be parsed as an individual field in splunk.

Subject:
Security ID: XXXXX\TEST01
Account Name: TEST01
Account Domain: TEST
Logon ID: 0xB7D860D0

Subject_Security_ID, Subject_Security_Account Name, Subject_Security _Account Domain and Subject_Security_LogonID

So kindly guide me on how to get this.

[Hemnaath]

Hi All,

Any help on this will be much appreciated !!!