hey xpac, we had downloaded from splunk base and renamed the add-on as per the naming standard ed followed in our organization. We have done very minor customization in it. This is the Props.conf stanza that are mapped to sourcetype=opsec. Props.conf: [opsec] KV_MODE = none TRANSFORMS-fix_am_sourcetype = opsec_sourcetype_anti_malware TRANSFORMS-fix_av_sourcetype = opsec_sourcetype_anti_virus TRANSFORMS-fix_sd_sourcetype = opsec_sourcetype_smartdefense TRANSFORMS-fix_te_sourcetype = opsec_sourcetype_threat_emulation TRANSFORMS-fix_vpn_sourcetype = opsec_sourcetype_vpn REPORT-0policy_id_tag_for_opsec = policy_id_tag_for_opsec, db_tag_for_opsec, mgmt_for_opsec, policy_date_for_opsec, policy_name_for_opsec REPORT-action_as_threat_emulation_action = action_as_threat_emulation_action REPORT-auto_kv_for_opsec = auto_kv_for_opsec REPORT-inbound_interface_for_opsec = inbound_interface_for_opsec REPORT-outbound_interface_for_opsec = outbound_interface_for_opsec REPORT-opsec_product = opsec_product_field,opsec_product REPORT-opsec_vendor_operation = opsec_vender_operation_field FIELDALIAS-dest_for_opsec = dst as dest, dst as dest_ip FIELDALIAS-src_for_opsec = src as src_ip FIELDALIAS-direction_for_opsec = i_f_dir as direction FIELDALIAS-dvc_for_opsec = orig as dvc FIELDALIAS-event_id_for_opsec = loc as event_id FIELDALIAS-vendor_product = product as vendor_product FIELDALIAS-vendor_action = action as vendor_action FIELDALIAS-dest_interface = outbound_interface as dest_interface FIELDALIAS-src_interface = inbound_interface as src_interface FIELDALIAS-icmp_code = ICMP as icmp_code FIELDALIAS-icmp_type = ICMP_Type as icmp_type FIELDALIAS-packets_in = client_inbound_packets as packets_in FIELDALIAS-packets_out = client_outbound_packets as packets_out FIELDALIAS-src_zone = inzone as src_zone FIELDALIAS-dest_zone = outzone as dest_zone EVAL-dvc_ip = if(match(orig,"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$"), orig, null) EVAL-rule = case(isnull(rule),policy_name,(not isnum(rule)),rule,isnotnull(policy_name),policy_name,1==1,null) EVAL-rule_id = if(isnum(rule),rule,null) EVAL-src_port = if(isnum(s_port),s_port,null) EVAL-src_svc = if(isnum(s_port),null,s_port) EVAL-dest_port = if(isnum(service),service,null) EVAL-dest_svc = if(isnum(service),null,service) EVAL-transport = case(lower(proto)=="tcp","tcp",lower(proto)=="udp","udp",lower(proto)=="icmp","icmp",1==1,null) EVAL-received_bytes = if (received_bytes < 0, 2147483647, received_bytes) EVAL-sent_bytes = if(sent_bytes < 0, 2147483647, sent_bytes) EVAL-bytes_in = if (received_bytes < 0, 2147483647, received_bytes) EVAL-bytes_out = if(sent_bytes < 0, 2147483647, sent_bytes) EVAL-bytes = if(bytes < 0, 2147483647, bytes) EVAL-client_inbound_bytes = if(client_inbound_bytes < 0, 2147483647, client_inbound_bytes) EVAL-client_outbound_bytes = if(client_outbound_bytes < 0, 2147483647,client_outbound_bytes) EVAL-server_inbound_bytes = if(server_inbound_bytes < 0, 2147483647, server_inbound_bytes) EVAL-server_outbound_bytes = if(server_outbound_bytes < 0, 2147483647,server_outbound_bytes) LOOKUP-action_for_opsec = checkpoint_opsec_action_lookup vendor_action OUTPUT action LOOKUP-vendor_info_for_opsec = checkpoint_vendor_info_lookup sourcetype OUTPUTNEW vendor,product,vendor_product LOOKUP-vendor_info_for_opsec2 = checkpoint_vendor_info_lookup sourcetype OUTPUTNEW product as opsec_product Kindly guide me how to fix the issue. thanks in advance. ... View more

xpac, I have gone through the Props.conf document but not sure how to begin. So Could you please guide me from where I need to start the stanza and how to create the stanza based on the requirement. link: https://docs.splunk.com/Documentation/Splunk/7.1.0/Admin/Propsconf#Field_extraction_configuration thanks in advance. ... View more

thanks dstaulcu, I am trying to seek help from xpac on this. ... View more

thanks xpac, for your time on this. I could see the below stanza details in the props.conf, but not sure where and what exactly needs to be created. Props.conf Details: ###### Windows Security Event Log ###### [source::*:Security] ## action LOOKUP-action_for_windows1_security = windows_action_lookup Type OUTPUTNEW action LOOKUP-action_for_windows2_security = windows_action_lookup Type as Keywords OUTPUTNEW action ## privilege REPORT-0vendor_privilege_for_windows_security = vendor_privilege_sv_for_windows_security,vendor_privilege_mv_for_windows_security REPORT-privilege_id_for_windows_security = privilege_id_for_windows_security LOOKUP-privilege_for_windows_security = windows_privilege_lookup privilege_id OUTPUT privilege FIELDALIAS-src_port_for_windows_security = Source_Port as src_port REPORT-Token_Elevation_Type_id_for_windows_security = Token_Elevation_Type_id_for_windows_security LOOKUP-vendor_info_for_windows_security = windows_vendor_info_lookup sourcetype OUTPUT vendor,product ## Set the app field to "win:remote" or "win:local" based on EventCode, Source_Network_Address, Target_Server_Name or Logon_Type LOOKUP-app0_for_windows_security = windows_app_lookup EventCode OUTPUTNEW app LOOKUP-app1_for_windows_security = windows_app_lookup Source_Network_Address OUTPUTNEW app LOOKUP-app2_for_windows_security = windows_app_lookup Target_Server_Name OUTPUTNEW app LOOKUP-app3_for_windows_security = windows_app_lookup Logon_Type OUTPUTNEW app LOOKUP-app4_for_windows_security = windows_app_lookup sourcetype OUTPUTNEW app ## Set the following fields based on order of operations REPORT-dest_for_windows_security = Target_Server_Name_as_dest,ComputerName_as_dest REPORT-dest_nt_domain_for_windows_security = Target_Domain_as_dest_nt_domain,Primary_Domain_as_dest_nt_domain,Account_Domain_as_dest_nt_domain,New_Domain_as_dest_nt_domain,Domain_as_dest_nt_domain REPORT-dest_nt_host_for_windows_security = Target_Server_Name_as_dest_nt_host,ComputerName_as_dest_nt_host REPORT-src_for_windows_security = Workstation_Name_as_src,Caller_Machine_Name_as_src,Client_Machine_Name_as_src,Source_Network_Address_as_src,Client_Address_as_src,ComputerName_as_src REPORT-src_ip_for_windows_security = Source_Network_Address_as_src_ip,Client_Address_as_src_ip REPORT-src_nt_domain_for_windows_security = Caller_Domain_as_src_nt_domain,Client_Domain_as_src_nt_domain,Account_Domain_as_src_nt_domain REPORT-src_nt_host_for_windows_security = Workstation_Name_as_src_nt_host,Caller_Machine_Name_as_src_nt_host,Client_Machine_Name_as_src_nt_host REPORT-src_user_for_windows_security = Caller_User_Name_as_src_user,Client_User_Name_as_src_user,Account_Name_as_src_user,User_Name_as_src_user REPORT-user_for_windows_security = Target_User_Name_as_user,Primary_User_Name_as_user,Target_Account_Name_as_user,New_Account_Name_as_user,Account_Name_as_user,User_Name_as_user,User_as_user Yes, I have gone through the Props.conf document but not sure how to start. So Could you please guide me from where I need to start the stanza and how to create the stanza based on the requirement. It will be great if you can provide me an example so that I can try myself first. thanks in advance. ... View more

thanks dstaulcu for your effort on this. Actually my client wants to get this filed value auto extracted instead executing the query by the end users. So could you please guide me on how to create a props & transforms stanza in order to get above same results automatically, with out executing the customized query. thanks in advance. ... View more

Hi All, Any help on this will be much appreciated !!! ... View more

xpac, thanks for your response, but i am not sure whether you had understood the requirement correctly or not. we need below details to be parsed as an individual field in splunk. Subject: Security ID: XXXXX\TEST01 Account Name: TEST01 Account Domain: TEST Logon ID: 0xB7D860D0 Subject_Security_ID, Subject_Security_Account Name, Subject_Security _Account Domain and Subject_Security_LogonID So kindly guide me on how to get this. ... View more

thanks sves for you are update. ... View more

I had followed the below link to achieve this http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Manageknowledgeobjectpermissions#Enable_a_role_other_than_Admin_and_Power_to_set_permissions_and_share_objects[link text]1 ... View more

Frank thanks for guiding me on this problem, Yes after updating the white list stanza in inputs.conf, we could limit/restrict only the particular events data getting ingested into splunk. Steps: 1) Edited the inputs.conf with the below stanza details. [WinEventLog://Windows PowerShell] disabled = 0 ignoreOlderThan = 2d current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 whitelist=800 index = windows [WinEventLog://Microsoft-Windows-PowerShell/Operational] disabled = 0 ignoreOlderThan = 2d current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 whitelist = 4103,4104 index = windows 2) After updating the inputs.conf stanza, it was deployed to all the universal forwarder via deployment server, by executing the ./splunk reload deploy-server. 3) Validated by executing the simple splunk query. index=windows sourcetype="WinEventLog:Microsoft-Windows-PowerShell/Operational" EventCode!=4103 AND EventCode!=4104 | dedup host EventCode | table host EventCode index=windows sourcetype="WinEventLog:Windows PowerShell" EventCode!=800 | dedup host EventCode | table host EventCode 4) By executing the above query got zero result. ... View more

Frank you are right, I had pushed the changes around 10:20 EDT and validate the same by executing the below query with time set from 10:40 EDT to Current time 13:20 EDT and found that its working perfectly, but could see only two of the host having an issue, when further investigated, noticed that app was not update, instead it was having an Error. Query Details: index=windows sourcetype="WinEventLog:Microsoft-Windows-PowerShell/Operational" EventCode!=4103 AND EventCode!=4104 | dedup host EventCode | table host EventCode index=windows sourcetype="WinEventLog:Windows PowerShell" EventCode!=800 | dedup host EventCode | table host EventCode Host having an issue in getting the update from DP instances. 05-02-2018 13:28:21.783 -0400 WARN ClientSessionsManager - ip=10.x.x.x name=9C691DDB-AF4A-4F10-90DB-94341B304F91 Updating record for sc=Test-SC-all_windows app=Test-IA-windows: action=Install result=Fail checksum=1927475316208885921 Anyway I will monitor for some more time and update you the result. ... View more

Frank, the above syntax did not work, we are able to see other EventCodes are also being ingested into splunk indexer instance, so kindly guide me on this. ... View more

Not yet pushed as we have almost 2000 + RM nodes configured in our environment, so need to get an approval to push this syntax and validate it. ... View more

thanks for your instant response on this, Frank we want only these EventCodes into Splunk not other EventCodes which currently getting ingested into splunk. [WinEventLog://Windows PowerShell] disabled = 0 ignoreOlderThan = 2d current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 whitelist=800 index = windows [WinEventLog://Microsoft-Windows-PowerShell/Operational] disabled = 0 ignoreOlderThan = 2d current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 whitelist = 4103,4104 index = windows Kindly guide me on this. ... View more

Hi Frank, hey can I use the below syntax to include only these EventCode 800, 4103 & 4104 event data from these two sourcetype in the inputs.conf. [WinEventLog://Windows PowerShell] disabled = 0 ignoreOlderThan = 2d current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 whitelist= EventCode=800 index = windows [WinEventLog://Microsoft-Windows-PowerShell/Operational] disabled = 0 ignoreOlderThan = 2d current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 whitelist= Eventcode=4103 whitelist1= Eventcode=4104 index = windows Got a doubt on this syntax as per the Splunk document the syntax provide for white listing the EventCode. syntax whitelist = key= link to the document: http://docs.splunk.com/Documentation/Splunk/7.1.0/Data/MonitorWindowseventlogdata#Create_advanced_filters_with_.27whitelist.27_and_.27blacklist.27 But in the above syntax we have not provide any regex details, so will this work. I mean by pushing this syntax will I able to include only with the event data containing the Event Code 800,4103 & 4104 in to splunk, not all other EventCodes. Kindly guide me on this. ... View more

thanks FrankVI, for much need a help on this problem, I am working on this for more than five hours with the above stanza , but unable to limit the eventcodes. When test with the above stanza I got the below error detail from splunkd.log,so not sure whether above stanza are correct or not correct me if it is wrong. ERROR Details: 04-26-2018 16:55:13.883 +0530 ERROR regexExtractionProcessor - REGEX field must be specified tranform_name=PowerShellnull host = lab.thay.com source = /opt/splunk/var/log/splunk/splunkd.log sourcetype =splunkd Anyway let me go through the inputs.conf document for whistling only few EventCodes. thanks for your much needed help 🙂 ... View more

Hey I had gone through referred splunk document link and created a stanza based on example provide in the document but It is not working as expected. Could any one guide me on the stanza to limit data to the indexer instance. thanks in advance. ... View more

hey can anyone guide me on the below stanza. we have tested the regex and its working fine, we are able to filter only the EventCode=800, 4013, 4014 by using the respective sourcetype. index=win sourcetype="WinEventLog:Microsoft-Windows-PowerShell/Operational" | regex "(?m)^EventCode=(4103|4104)" index=win sourcetype="WinEventLog:Windows PowerShell" | regex "(?m)^EventCode=800" kindly guide me whether the above props & transforms.conf stanza are good to implement. ... View more

thank gurav for sharing the link, but can you please correct me whether the below stanza can resolve my requirement. Currently we need only the below EventCodes not the other eventcodes. sourcetype="WinEventLog:Microsoft-Windows-PowerShell/Operational" LogName=Microsoft-Windows-PowerShell/Operational SourceName=Microsoft-Windows-PowerShell EventCode=4103 EventType=4 sourcetype="WinEventLog:Microsoft-Windows-PowerShell/Operational" LogName=Microsoft-Windows-PowerShell/Operational SourceName=Microsoft-Windows-PowerShell EventCode=4104 EventType=5 sourcetype="WinEventLog:Windows LogName=Windows PowerShell SourceName=PowerShell EventCode=800 EventType=4 Kindly guide me on this. ... View more

thanks Frank, for providing the query to fetch the event count based on per host. Based on that, I could see that two of the host has maximum event count for a duration of 1 hour. For the sourcetype: opsec index=firewall test01fw = 190845 For the sourcetype : infoblox:dns and index=network host01 = 217078 These two host had maximum spike from the visualization chart. What should be next level of investigation should be done to find out the cause for this spike in log volume. ... View more

I had used the below query to find out the event count details from the two sourcetype opsec/ infoblox:dns and found event count details Query 1: index=_internal source=license_usage.log type="Usage" idx=* st="infoblox:dns" | stats sum(gb) as Totalcount by st,idx | sort - Totalcount | eventstats sum(Totalcount) as SUM | eval P=round(((Totalcount/SUM)*100),2) | eval Percentage=P+" "+"%" Total Number of events =3856 for a 1 hour duration Query 2: index=_internal source=license_usage.log type="Usage" idx=* st="opsec" | stats sum(gb) as Totalcount by st,idx | sort - Totalcount | eventstats sum(Totalcount) as SUM | eval P=round(((Totalcount/SUM)*100),2) | eval Percentage=P+" "+"%" Total Number of events =1475 for a 1 hour duration Kindly correct me if this is not what you had comment above. ... View more

Frank, how to use the tstats command for fetching the details which had commented in your previous comment. "by simple tstats search on the respective index(es), to figure out whether perhaps it is a certain specific data source (host) spiking" It will be great if can give me SPL query using tstats command. ... View more