topic Re: SEP 14.2 RU1 log format change in All Apps and Add-ons

SEP 14.2 RU1 log format change

jtwind_2 — Wed, 30 Sep 2020 00:30:07 GMT

Symantec slightly change the log format for 14.2 RU1... add these to transforms.conf in /local and you'll be good to go.

[field_extraction_for_traffic]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Local Host:\s*(?[[sep_file_field]]))?,\s*(?:Local Port:\s*(?[[sep_file_field]]))?,\s*(?:Local Host MAC:\s*(?[[sep_file_field]]))?,\s*(?:Remote Host IP:\s*(?[[sep_file_field]]))?,\s*(?:Remote Host Name:\s*(?[[sep_file_field]]))?,\s*(?:Remote Port:\s*(?[[sep_file_field]]))?,\s*(?:Remote Host MAC:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Begin:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?[[sep_file_field]]))?,\s*(?:Application:\s*(?[[sep_file_field]]))?,\s*(?:Rule:\s*(?[[sep_file_field]]))?,\s*(?:Location:\s*(?[[sep_file_field]]))?,\s*(?:User:\s*(?[[sep_file_field]]))?,\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Action:\s*(?[[sep_file_field]]))?,\s*(?:SHA-256:\s*(?[[sep_file_field]]))?,\s*(?:MD-5:\s*(?[[sep_file_field]]))?

[field_extraction_for_agt_security]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Event Description:\s*(?[[sep_file_field]])),\s*(?:Local:\s*(?[[sep_file_field]]))?,\s*(?:Local Host MAC:\s*(?[[sep_file_field]]))?,\s*(?:Remote Host Name:\s*(?[[sep_file_field]]))?,\s*(?:Remote Host IP:\s*(?[[sep_file_field]]))?,\s*(?:Remote Host MAC:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Begin:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?[[sep_file_field]]))?,\s*(?:Application:\s*(?[[sep_file_field]]))?,\s*(?:Location:\s*(?[[sep_file_field]]))?,\s*(?:User:\s*(?[[sep_file_field]])),\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Local\sPort\s*(?[[sep_file_field]]))?,\s*(?:Remote\sPort\s*(?[[sep_file_field]]))?,\s*(?:CIDS\sSignature\sID:\s*(?[[sep_file_field]]))?,\s*(?:CIDS\sSignature\sstring:\s*(?[[sep_file_field]]))?,\s*(?:CIDS\sSignature\sSubID:\s*(?[[sep_file_field]]))?,\s*(?:Intrusion\sURL:\s*(?[[sep_file_field]]))?,\s*(?:Intrusion\sPayload\sURL:\s*(?[[sep_file_field]]))?,?\s*(?:SHA-256:\s*(?[[sep_file_field]]))?,?\s*(?:MD-5:\s*(?[[sep_file_field]]))?

[field_extraction_for_agt_risk]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?[^,']'[^']'|[^,"]"[^"]|[^,]))?,\s(?:Source:\s*(?[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Actual\saction:\s*(?[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?[[sep_file_field]]))?,s*(?:Inserted:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?[[sep_file_field]]))?,\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Group:\s*(?[[sep_file_field]]))?,\s*(?:Server:\s*(?[[sep_file_field]]))?,\s*(?:User:\s*(?[[sep_file_field]])),\s*(?:Source\scomputer:\s*(?[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?[[sep_file_field]]))?,\s*(?:Disposition:\s*(?[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?.))?,\s(?:Downloaded\sby:\s*(?[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?[[sep_file_field]]))?,\s*(?:Confidence:\s*(?[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?:Application\shash:\s*(?[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?.))?,\s(?:Application\sname:\s(?[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P.))?,\s(?:Application\stype:\s*(?[[sep_file_field]]))?,\s*(?:File\ssize\s(bytes):\s*(?[[sep_file_field]]))?(?:,\s*Category\sset:\s*(?[[sep_file_field]]),\s*Category\stype:\s*(?[[sep_file_field]]))?,?\s*(?:Location:\s*(?[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?[[sep_file_field]]))?

[field_extraction_for_agt_behavior]
REGEX = ^(?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),?\s*(?[[sep_file_field]])?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Begin:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Rule:\s*(?[[sep_file_field]])),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:User:\s*(?[[sep_file_field]])),\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Action\sType:\s*(?[[sep_file_field]]))?(?:,\s*File\ssize\s(bytes):\s*(?[[sep_file_field]]),\s*Device\sID:\s*(?[[sep_file_field]]))?$

[field_extraction_for_agt_proactive]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?:Computer\sname:\s*(?[[sep_file_field]]))?,?\s*(?:IP\sAddress:\s*(?[[sep_file_field]]))?,\s*(?:Detection\stype:\s*(?[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?[[sep_file_field]]))?,\s*(?:Application\sname:\s*(?[[sep_file_field]]))?,\s*(?:Application\stype:\s*(?[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?[[sep_file_field]]))?,\s*(?:Application\shash:\s*(?[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?.))?,\s(?:File\ssize\s(bytes):\s*(?[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?[[sep_file_field]]))?,\s*(?:Detection\sscore:\s*(?[[sep_file_field]]))?,\s*(?:COH\sEngine\sVersion:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?:Permitted\sapplication\sreason:\s*(?[[sep_file_field]]))?,\s*(?:Disposition:\s*(?[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?.))?,\s(?:Downloaded\sby:\s*(?[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?[[sep_file_field]]))?,\s*(?:Confidence:\s*(?[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?[[sep_file_field]]))?,\s*(?:Risk\sLevel:\s*(?[[sep_file_field]]))?,?\s*(?:Risk\stype:\s*(?[[sep_file_field]]))?,?\s*(?:Detection\sSource:\s*(?[[sep_file_field]]))?,\s*(?:Source:\s*(?[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Actual\saction:\s*(?[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?[[sep_file_field]]))?,\s*(?:Inserted:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Group:\s*(?[[sep_file_field]]))?,\s*(?:Server:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?:Source\scomputer:\s*(?[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?[[sep_file_field]]))?

Re: SEP 14.2 RU1 log format change

GDustin — Fri, 17 May 2019 09:28:16 GMT

what version does this work on?

I am getting this on 7.0.3;

Bad regex value: ... / REGEX; why: unrecognized character after (? or (?-
for all those stanzas

Re: SEP 14.2 RU1 log format change

GDustin — Fri, 17 May 2019 10:57:32 GMT

@jtwind_2
Can you repost with the Code sample option?

This is from the old 2.3.0 transforms[as an example];

(?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?<Computer_Name>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,\s*(?:Inserted:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?<Last_Update_Time>[[sep_file_field]]))?,\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?<user>[[sep_file_field]]),\s*(?:Source\scomputer:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?<Unknown_Field>[[sep_file_field]]),\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?<Reason_For_White_Listing>[[sep_file_field]]),\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:Application\sname:\s(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P<Application_Version>.*))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?(?:,\s*Category\sset:\s*(?<Category_Set>[[sep_file_field]]),\s*Category\stype:\s*(?<Category_Type>[[sep_file_field]]))?,?\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?

Re: SEP 14.2 RU1 log format change

rriegert — Mon, 20 May 2019 17:45:44 GMT

I am also getting the Bad Regex error, but I'm confused by the latest post. I tried adding the (or whatever the verbiage was between the <> in the original transforms Regex per the applicable stanza) in the local file regex where it had been in the original transforms Regex, but am still getting the error. I'm unsure if this was the recommendation, or what additional regex tweaking needs to be performed. Anyone know if Splunk will be putting out an updated TA for these critical parsing changes?

Re: SEP 14.2 RU1 log format change

jtwind_2 — Mon, 20 May 2019 17:51:43 GMT

Try this:... you'll also want to be running v 2.3.0 (latest as of this writing) of the Symantec add-on... https://splunkbase.splunk.com/app/2772/

[field_extraction_for_traffic]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?<vendor_severity>[[sep_file_field]]),\s*(?<Host_Name>[[sep_file_field]]),\s*(?:Local Host:\s*(?<Local_Host_IP>[[sep_file_field]]))?,\s*(?:Local Port:\s*(?<Local_Port>[[sep_file_field]]))?,\s*(?:Local Host MAC:\s*(?<Local_Host_MAC>[[sep_file_field]]))?,\s*(?:Remote Host IP:\s*(?<Remote_Host_IP>[[sep_file_field]]))?,\s*(?:Remote Host Name:\s*(?<Remote_Host_Name>[[sep_file_field]]))?,\s*(?:Remote Port:\s*(?<Remote_Port>[[sep_file_field]]))?,\s*(?:Remote Host MAC:\s*(?<Remote_Host_MAC>[[sep_file_field]]))?,\s*(?<Network_Protocol>[[sep_file_field]]),\s*(?<Traffic_Direction>[[sep_file_field]]),\s*(?:Begin:\s*(?<Begin_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?:Application:\s*(?<Application_Name>[[sep_file_field]]))?,\s*(?:Rule:\s*(?<rule>[[sep_file_field]]))?,\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?,\s*(?:User:\s*(?<user>[[sep_file_field]]))?,\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Action:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:SHA-256:\s*(?<SHA_256>[[sep_file_field]]))?,\s*(?:MD-5:\s*(?<MD_5>[[sep_file_field]]))?

[field_extraction_for_agt_security]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?<vendor_severity>[[sep_file_field]]),\s*(?<Host_Name>[[sep_file_field]]),\s*(?:Event Description:\s*(?<Event_Description>[[sep_file_field]])),\s*(?:Local:\s*(?<Local_Host_IP>[[sep_file_field]]))?,\s*(?:Local Host MAC:\s*(?<Local_Host_MAC>[[sep_file_field]]))?,\s*(?:Remote Host Name:\s*(?<Remote_Host_Name>[[sep_file_field]]))?,\s*(?:Remote Host IP:\s*(?<Remote_Host_IP>[[sep_file_field]]))?,\s*(?:Remote Host MAC:\s*(?<Remote_Host_MAC>[[sep_file_field]]))?,\s*(?<Traffic_Direction>[[sep_file_field]]),\s*(?<Network_Protocol>[[sep_file_field]]),\s*(?<Hack_Type>[[sep_file_field]]),\s*(?:Begin:\s*(?<Begin_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?:Application:\s*(?<Application_Name>[[sep_file_field]]))?,\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?,\s*(?:User:\s*(?<user>[[sep_file_field]])),\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Local\sPort\s*(?<Local_Port>[[sep_file_field]]))?,\s*(?:Remote\sPort\s*(?<Remote_Port>[[sep_file_field]]))?,\s*(?:CIDS\sSignature\sID:\s*(?<CIDS_Signature_ID>[[sep_file_field]]))?,\s*(?:CIDS\sSignature\sstring:\s*(?<CIDS_Signature_String>[[sep_file_field]]))?,\s*(?:CIDS\sSignature\sSubID:\s*(?<CIDS_Signature_SubID>[[sep_file_field]]))?,\s*(?:Intrusion\sURL:\s*(?<Intrusion_URL>[[sep_file_field]]))?,\s*(?:Intrusion\sPayload\sURL:\s*(?<Intrusion_Payload_URL>[[sep_file_field]]))?,?\s*(?:SHA-256:\s*(?<SHA_256>[[sep_file_field]]))?,?\s*(?:MD-5:\s*(?<MD_5>[[sep_file_field]]))?

[field_extraction_for_agt_risk]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?<Computer_Name>[^,']*'[^']*'|[^,"]*"[^"]*|[^,]*))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,s*(?:Inserted:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?<Last_Update_Time>[[sep_file_field]]))?,\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?:User:\s*(?<user>[[sep_file_field]])),\s*(?:Source\scomputer:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?<Reason_For_White_Listing>[[sep_file_field]]),\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:Application\sname:\s(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P<Application_Version>.*))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?(?:,\s*Category\sset:\s*(?<Category_Set>[[sep_file_field]]),\s*Category\stype:\s*(?<Category_Type>[[sep_file_field]]))?,?\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))?

[field_extraction_for_agt_behavior]
REGEX = ^(?i)(?:[[sep_file_prefix]]),\s*(?<vendor_severity>[[sep_file_field]]),\s*(?<Host_Name>[[sep_file_field]]),?\s*(?<IP_Address>[[sep_file_field]])?,\s*(?<vendor_action>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?<API>[[sep_file_field]]),\s*(?:Begin:\s*(?<Begin_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Rule:\s*(?<rule>[[sep_file_field]])),\s*(?<Caller_Process_ID>[[sep_file_field]]),\s*(?<Caller_Process_Name>[[sep_file_field]]),\s*(?<Return_Address>[[sep_file_field]]),\s*(?<Return_Module>[[sep_file_field]]),\s*(?<Parameter>[[sep_file_field]]),\s*(?:User:\s*(?<user>[[sep_file_field]])),\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Action\sType:\s*(?<Action_Type>[[sep_file_field]]))?(?:,\s*File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]),\s*Device\sID:\s*(?<Device_ID>[[sep_file_field]]))?$

[field_extraction_for_agt_proactive]
REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:Computer\sname:\s*(?<Computer_Name>[[sep_file_field]]))?,?\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Detection\stype:\s*(?<Detection_Type>[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Application\sname:\s*(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?<Application_Version>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?:Detection\sscore:\s*(?<Detection_Score>[[sep_file_field]]))?,\s*(?:COH\sEngine\sVersion:\s*(?<COH_Engine_Version>[[sep_file_field]]))?,\s*(?<Submission_Recommendation>[[sep_file_field]]),\s*(?:Permitted\sapplication\sreason:\s*(?<Permitted_Application_Reason>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?:Risk\sLevel:\s*(?<Risk_Level>[[sep_file_field]]))?,?\s*(?:Risk\stype:\s*(?<Risk_Type>[[sep_file_field]]))?,?\s*(?:Detection\sSource:\s*(?<Detection_Source>[[sep_file_field]]))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,\s*(?:Inserted:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?<user>[[sep_file_field]]),\s*(?:Source\scomputer:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))?

Re: SEP 14.2 RU1 log format change

GDustin — Wed, 30 Sep 2020 00:38:44 GMT

@rriegert see below, jt come through;

attempt to reduce confusion:
Paste old code in something like regex101

Take the smallest one as an example:
[field_extraction_for_agt_behavior]

^(?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),?\s*(?[[sep_file_field]])?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Begin:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Rule:\s*(?[[sep_file_field]])),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:User:\s*(?[[sep_file_field]])),\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Action\sType:\s*(?[[sep_file_field]]))?(?:,\s*File\ssize\s(bytes):\s*(?[[sep_file_field]]),\s*Device\sID:\s*(?[[sep_file_field]]))?$

Get these errors:
All the errors detected are listed below, from left to right, as they appear in the pattern.
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
? The preceding token is not quantifiable
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure
(? Incomplete group structure
) Incomplete group structure

The problem is when you paste a code such as:
?
Without the code option in the answers post, the:

get's filtered or removed;

Now; Take the smallest one as an example below; and go paste that one in regex101 or similar:
[field_extraction_for_agt_behavior]

^(?i)(?:[[sep_file_prefix]]),\s*(?<vendor_severity>[[sep_file_field]]),\s*(?<Host_Name>[[sep_file_field]]),?\s*(?<IP_Address>[[sep_file_field]])?,\s*(?<vendor_action>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?<API>[[sep_file_field]]),\s*(?:Begin:\s*(?<Begin_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Rule:\s*(?<rule>[[sep_file_field]])),\s*(?<Caller_Process_ID>[[sep_file_field]]),\s*(?<Caller_Process_Name>[[sep_file_field]]),\s*(?<Return_Address>[[sep_file_field]]),\s*(?<Return_Module>[[sep_file_field]]),\s*(?<Parameter>[[sep_file_field]]),\s*(?:User:\s*(?<user>[[sep_file_field]])),\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Action\sType:\s*(?<Action_Type>[[sep_file_field]]))?(?:,\s*File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]),\s*Device\sID:\s*(?<Device_ID>[[sep_file_field]]))?$

...no errors

Re: SEP 14.2 RU1 log format change

GDustin — Tue, 21 May 2019 07:25:47 GMT

So far worked in dev with a one-shot; looking good for prod;Thank you for sharing.

Re: SEP 14.2 RU1 log format change

GDustin — Wed, 22 May 2019 23:14:43 GMT

some of my code above is still bad/not displaying correctly; it is not presenting the characters "<" or ">", sorry for the confusion

Re: SEP 14.2 RU1 log format change

archme — Wed, 30 Sep 2020 00:54:06 GMT

i am using the following for my agt_risk extraction:

[field_extraction_for_agt_risk]

REGEX =(?i)(?:[[sep_file_prefix]]),\s*(?[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?[[sep_file_field]]))?,\s*(?:Source:\s*(?[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?[[sep_file_field]]),\s*(?:Actual\saction:\s*(?[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?[[sep_file_field]]))?,\s*(?:Inserted:\s*(?[[sep_file_field]]))?,\s*(?:End:\s*(?[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?[[sep_file_field]]))?,\s*(?:Domain:\s*(?[[sep_file_field]]))?,\s*(?:Group:\s*(?[[sep_file_field]]))?,\s*(?:Server:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?:Source\scomputer:\s*(?[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?[[sep_file_field]]))?,\s*(?:Disposition:\s*(?[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?[[sep_file_field]]))?,\s*(?:Downloaded\sby:\s*(?[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?[[sep_file_field]]))?,\s*(?:Confidence:\s*(?[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?[[sep_file_field]]))?,\s*(?[[sep_file_field]]),\s*(?:Application\shash:\s*(?[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?.))?,\s(?:Application\sname:\s(?[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P[[sep_file_field]]))?,\s*(?:Application\stype:\s*(?[[sep_file_field]]))?,\s*(?:File\ssize\s(bytes):\s*(?[[sep_file_field]]))?,\s*(?:Category\sset:\s*(?[[sep_file_field]]))?,\s*(?:Category\stype:\s*(?[[sep_file_field]]))?,?\s*(?:Location:\s*(?[[sep_file_field]]))?,\s*(?:Intensive\sProtection\sLevel:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?[[sep_file_field]]))?

Re: SEP 14.2 RU1 log format change

testrake_trek — Wed, 31 Jul 2019 17:36:39 GMT

This worked for me as well. In Splunk Cloud - updated the configuration via the webui and refreshed the searches. Thank you.

Re: SEP 14.2 RU1 log format change

csperry_splunk — Wed, 31 Jul 2019 18:42:08 GMT

We took a different approach as the transforms option gave us problems when not all the fields existed all this time in the events we were getting. As such we updated the local/props.conf with the the below and haven't had any problem reported yet:

[symantec:ep:security:file]
EXTRACT-security_file_fields = \[name\]:(?P<name>.+?)\[class\]:(?P<class>.+?)\[guid\]:(?P<guid>.+?)\[deviceID\]:(?P<deviceID>.+)[\\\\](?P<deviceSN>.+?)\,

[symantec:ep:agents:db]
FIELDALIAS-user = CURRENT_LOGIN_USER AS user
FIELDALIAS-dest = COMPUTER_NAME AS dest
FIELDALIAS-ip = ip_address AS dest_ip
FIELDALIAS-dest_mac = mac_address AS dest_mac
FIELDALIAS-domain = domain_name AS dest_nt_domain
FIELDALIAS-product_ver = AGENT_VERSION AS product_version
FIELDALIAS-signature_ver = AV_REVISION AS signature_version
EVAL-vendor = "Symantec"
EVAL-product = "Endpoint Protection"
EVAL-vendor_product = "Symantec Endpoint Protection"

[symantec:ep:proactive:file]
EXTRACT-proactive_downloaded_by = Downloaded\sby\:\s(?<Downloaded_By>.*?[^\,]*)
EXTRACT-proactive_prevalance = Prevalence\:\s(?<Prevalence>.*?[^\,]*)
EXTRACT-proactive_url_track = URL\sTracking\sStatus\:\s(?<URL_Tracking_Status>.*?[^\,]*)
EXTRACT-proactive_first_seen = First\sSeen\:\s(?<First_Seen>.*?[^\,]*)
EXTRACT-proactive_sensitivity = Sensitivity\:\s(?<Sensitivity>.*?[^\,]*)
EXTRACT-proactive_app_hash = Application\shash\:\s(?<Application_Hash>.*?[^\,]*)
EXTRACT-proactive_hash_type = Hash\stype\:\s(?<Hash_Type>.*?[^\,]*)
EXTRACT-proactive_app_name = Application\sname\:\s(?<Application_Name>.*?[^\,]*) 
EXTRACT-proactive_app_ver = Application\sversion\:\s(?<Application_Version>.*?[^\,]*)
EXTRACT-proactive_app_type = Application\stype\:\s(?<Application_Type>.*?[^\,]*)
EXTRACT-proactive_file_size = File\ssize\s\(bytes\)\:\s(?<File_Size>.*?[^\,]*)
EXTRACT-proactive_location = Location\:\s(?<Location>.*?[^\,]*)
EXTRACT-proactive_intensive_protection_lvl = Intensive\sProtection\sLevel\:\s(?<Intensive_Protection_Level>.*?[^\,]*)
EXTRACT-proactive_cert_issuer = Certificate\sissuer\:\s(?<Certificate_Issuer>.*?[^\,]*)
EXTRACT-proactive_cert_signer = Certificate\ssigner\:\s(?<Certificate_Signer>.*?[^\,]*)
EXTRACT-proactive_cert_thumbprint = Certificate\sthumbprint\:\s(?<Certificate_Thumbprint>.*?[^\,]*)
EXTRACT-proactive_signing_timestamp = Signing\stimestamp\:\s(?<Signing_Timestamp>.*?[^\,]*)
EXTRACT-proactive_cert_serial_no = Certificate\sserial\snumber\:\s(?<Certificate_Serial_Number>.*?[^\,]*)
EXTRACT-proactive_ip = IP\sAddress\:\s+(?<IP_Address>\d[^\,]+)
EXTRACT-proactive_comp_name = Computer\sname\:\s(?<Computer_Name>\w[^\,]+)
EXTRACT-proactive_src = Source\:\s(?<Source>\w[^\,]+)
EXTRACT-proactive_name = Risk\sname\:\s(?<Risk_Name>\w[^\,]+)
EXTRACT-proactive_occurrences = Occurrences\:\s+(?<Occurrences>\d[^\,]*)\,(?<file_path>\w[^\,]+)\,(?<Description>\w*)
EXTRACT-proactive_actual_action = Actual\saction\:\s(?<vendor_action>\w[^\,]+)
EXTRACT-proactive_requested_action = Requested\saction\:\s(?<Requested_Action>\w[^\,]+)
EXTRACT-proactive_secondary_action = Secondary\saction\:\s(?<Secondary_Action>\w[^\,]+)
EXTRACT-proactive_event_time = Event\stime\:\s(?<Event_Time>\d[^\,]+)
EXTRACT-proactive_insert_time = Inserted\:\s(?<Event_Insert_Time>\d[^\,]+)
EXTRACT-proactive_end_time = End\:\s(?<End_Time>\d[^\,]+)
EXTRACT-proactive_domain_name = Domain\:\s(?<Domain_Name>\w[^\,]+)
EXTRACT-proactive_group_name = Group\:\s(?<Group_Name>\w[^\,]+)
EXTRACT-proactive_server_name = Server\:\s(?<Server_Name>\w[^\,]+)
EXTRACT-proactive_user_name = User\:\s(?<user>\w[^\,]+)
EXTRACT-proactive_src_name = Source\scomputer\:\s(?<Source_Computer_Name>.*?[^\,]*)
EXTRACT-proactive_src_ip = Source\sIP\:\s(?<Source_Computer_IP>.*?[^\,]*)
EXTRACT-proactive_disposition = Disposition\:\s(?<Disposition>\w[^\,]+)
EXTRACT-proactive_download_site = Download\ssite\:\s(?<Download_Site>.*?[^\,]*)
EXTRACT-proactive_web_domain = Web\sdomain\:\s(?<Web_Domain>.*?[^\,]*)
EXTRACT-proactive_confidence = Confidence\:\s(?<Confidence>.*?[^\,]*)
EXTRACT-proactive_action = ^[\d\-\s\:]+\,(?<Risk_Action>.*?[^\,]*)
EXTRACT-proactive_detection_type = Detection\stype\:\s+(?<Detection_Type>.*?[^\,]*)
EXTRACT-proactive_detection_score = Detection\sscore\:\s(?<Detection_Score>.*?[^\,]*)
EXTRACT-proactive_coh_engine_ver = COH\sEngine\sVersion\:\s(?<coh_engine_version>.*?[^\,]*)\,(?<Submission_Recommendation>.*?[^\,]*)
EXTRACT-proactive_permitted_app_reason = Permitted\sapplication\sreason\:\s(?<Permitted_Application_Reason>.*?[^\,]*)
EXTRACT-proactive_risk_lvl = Risk\sLevel\:\s(?<Risk_Level>.*?[^\,]*)
EXTRACT-proactive_risk_type = Risk\stype\:\s(?<Risk_Type>.*?[^\,]*)

[symantec:ep:risk:file]
EXTRACT-risk_downloaded_by = Downloaded\sby\:\s(?<Downloaded_By>.*?[^\,]*)
EXTRACT-risk_prevalance = Prevalence\:\s(?<Prevalence>.*?[^\,]*)
EXTRACT-risk_url_track = URL\sTracking\sStatus\:\s(?<URL_Tracking_Status>.*?[^\,]*)
EXTRACT-risk_first_seen = First\sSeen\:\s(?<First_Seen>.*?[^\,]*)
EXTRACT-risk_sensitivity = Sensitivity\:\s(?<Sensitivity>.*?[^\,]*)\,(?<Reason_For_White_Listing>.*?[^\,]*)
EXTRACT-risk_app_hash = Application\shash\:\s(?<Application_Hash>.*?[^\,]*)
EXTRACT-risk_hash_type = Hash\stype\:\s(?<Hash_Type>.*?[^\,]*)
EXTRACT-risk_co_name = Company\sname\:\s(?<Company_Name>.*?[^\,]*)
EXTRACT-risk_app_name = Application\sname\:\s(?<Application_Name>.*?[^\,]*) 
EXTRACT-risk_app_ver = Application\sversion\:\s(?<Application_Version>.*?[^\,]*)
EXTRACT-risk_app_type = Application\stype\:\s(?<Application_Type>.*?[^\,]*)
EXTRACT-risk_file_size = File\ssize\s\(bytes\)\:\s(?<File_Size>.*?[^\,]*)
EXTRACT-risk_cat_set = Category\sset\:\s(?<Category_Set>.*?[^\,]*)
EXTRACT-risk_cat_type = Category\stype\:\s(?<Category_Type>.*?[^\,]*)
EXTRACT-risk_location = Location\:\s(?<Location>.*?[^\,]*)
EXTRACT-risk_intensive_protection_lvl = Intensive\sProtection\sLevel\:\s(?<Intensive_Protection_Level>.*?[^\,]*)
EXTRACT-risk_cert_issuer = Certificate\sissuer\:\s(?<Certificate_Issuer>.*?[^\,]*)
EXTRACT-risk_cert_signer = Certificate\ssigner\:\s(?<Certificate_Signer>.*?[^\,]*)
EXTRACT-risk_cert_thumbprint = Certificate\sthumbprint\:\s(?<Certificate_Thumbprint>.*?[^\,]*)
EXTRACT-risk_signing_timestamp = Signing\stimestamp\:\s(?<Signing_Timestamp>.*?[^\,]*)
EXTRACT-risk_cert_serial_no = Certificate\sserial\snumber\:\s(?<Certificate_Serial_Number>.*?[^\,]*)
EXTRACT-risk_ip = IP\sAddress\:\s+(?<IP_Address>\d[^\,]+)
EXTRACT-risk_comp_name = Computer\sname\:\s(?<Computer_Name>\w[^\,]+)
EXTRACT-risk_src = Source\:\s(?<Source>\w[^\,]+)
EXTRACT-risk_name = Risk\sname\:\s(?<Risk_Name>\w[^\,]+)
EXTRACT-risk_occurrences = Occurrences\:\s+(?<Occurrences>\d[^\,]*)\,(?<file_path>\w[^\,]+)\,(?<Description>\w*)
EXTRACT-risk_actual_action = Actual\saction\:\s(?<vendor_action>\w[^\,]+)
EXTRACT-risk_requested_action = Requested\saction\:\s(?<Requested_Action>\w[^\,]+)
EXTRACT-risk_secondary_action = Secondary\saction\:\s(?<Secondary_Action>\w[^\,]+)
EXTRACT-risk_event_time = Event\stime\:\s(?<Event_Time>\d[^\,]+)
EXTRACT-risk_insert_time = Inserted\:\s(?<Event_Insert_Time>\d[^\,]+)
EXTRACT-risk_end_time = End\:\s(?<End_Time>\d[^\,]+)
EXTRACT-risk_update_time = Last\supdate\stime\:\s(?<Last_Update_Time>\d[^\,]+)
EXTRACT-risk_domain_name = Domain\:\s(?<Domain_Name>\w[^\,]+)
EXTRACT-risk_group_name = Group\:\s(?<Group_Name>\w[^\,]+)
EXTRACT-risk_server_name = Server\:\s(?<Server_Name>\w[^\,]+)
EXTRACT-risk_user_name = User\:\s(?<user>\w[^\,]+)
EXTRACT-risk_src_name = Source\scomputer\:\s(?<Source_Computer_Name>.*?[^\,]*)
EXTRACT-risk_src_ip = Source\sIP\:\s(?<Source_Computer_IP>.*?[^\,]*)
EXTRACT-risk_disposition = Disposition\:\s(?<Disposition>\w[^\,]+)
EXTRACT-risk_download_site = Download\ssite\:\s(?<Download_Site>.*?[^\,]*)
EXTRACT-risk_web_domain = Web\sdomain\:\s(?<Web_Domain>.*?[^\,]*)
EXTRACT-risk_confidence = Confidence\:\s(?<Confidence>.*?[^\,]*)
EXTRACT-risk_action = ^[\d\-\s\:]+\,(?<Risk_Action>.*?[^\,]*)

[symantec:ep:security:file]
EXTRACT-security_vendor_severity = ^[\d\-\s\:]+\,(?<vendor_severity>.*?[^\,]*)\,(?<Host_Name>\w[^\,]+)
EXTRACT-security_event_desc = Event\sDescription(.*?)(?:\"\,|\s\w+\:|\s+\[\w+\]\:)
EXTRACT-security_domain_name = Domain\:\s(?<Domain_Name>\w[^\,]+)
EXTRACT-security_location = Location\:\s(?<Location>.*?[^\,]*)
EXTRACT-security_begin_time = Begin\:\s(?<Begin_Time>\d[^\,]+)
EXTRACT-security_end_time = End\:\s(?<End_Time>\d[^\,]+)
EXTRACT-security_occurrences = Occurrences\:\s+(?<Occurrences>\d[^\,]*)
EXTRACT-security_user_name = User\:\s(?<user>\w[^\,]+)
EXTRACT-security_local_pt = Local\sPort\:\s+(?<Local_Port>\d[^\,]*)
EXTRACT-security_remote_pt = Remote\sPort\:\s+(?<Remote_Port>\d[^\,]*)
EXTRACT-security_local_ip = Local\:\s+(?<Local_Host_IP>\d[^\,]+)
EXTRACT-security_remote_name = Remote\s\Host\sName\:\s(?<Remote_Host_Name>.*?[^\,]*)
EXTRACT-security_remote_ip = Remote\sHost\sIP\:\s(?<Remote_Host_IP>\d[^\,]+)
EXTRACT-security_local_mac = Local\sHost\sMAC\:\s(?<Local_Host_MAC>\w[^\,]+)
EXTRACT-security_intrusion_url = Intrusion\sURL\:\s(?<Intrusion_URL>.*?[^\,]*)
EXTRACT-security_intrusion_payload_url = Intrusion\sPayload\sURL\:\s(?<Intrusion_Payload_URL>.*?[^\,]*)
EXTRACT-security_md5 = MD\-5\:\s(?<MD_5>.*?[^\,]*)
EXTRACT-security_sha256 = SHA\-256\:\s(?<SHA_256>.*?[^\,]*)
EXTRACT-security_signature_id = CIDS\sSignature\sID\:\s(?<CIDS_Signature_ID>.*?[^\,]*)
EXTRACT-security_signature_string = CIDS\sSignature\sstring\:\s(?<CIDS_Signature_String>.*?[^\,]*)
EXTRACT-security_signature_subid = CIDS\sSignature\sSubID\:\s(?<CIDS_Signature_SubID>.*?[^\,]*)
EXTRACT-security_app_name = Application\:\s(?<Application_Name>.*?[^\,]*)
EXTRACT-security_remote_mac = Remote\sHost\sMAC\:\s(?<Remote_Host_MAC>\d[^\,]+)\,(?<Traffic_Direction>\w[^\,]+)\,(?<Network_Protocol>\d[^\,]*)\,(?<Hack_Type>\w*)
EXTRACT-security_app_path = Application\spath\:\s(?<Application_Path>.*?[^\,]*)
EXTRACT-security_sid = \[SID\:\s(?<SID>\d[^\]]+)
EXTRACT-security_audit = Audit\:\s(?<Audit>.*?[^\,.]*)(?=.\s|\,)
EXTRACT-security_requirement = Requirement\:\s(?<Requirement1>.*?[^\,]*)\sRequirement\:\s(?<Requirement2>.*?[^\,]*)

[symantec:ep:traffic:file]
EXTRACT-traffic_vendor_severity = ^[\d\-\s\:]+\,(?<vendor_severity>.*?[^\,]*)\,(?<Host_Name>\w[^\,]+)
EXTRACT-traffic_domain_name = Domain\:\s(?<Domain_Name>\w[^\,]+)
EXTRACT-traffic_location = Location\:\s(?<Location>.*?[^\,]*)
EXTRACT-traffic_begin_time = Begin\:\s(?<Begin_Time>\d[^\,]+)
EXTRACT-traffic_end_time = End\:\s(?<End_Time>\d[^\,]+)
EXTRACT-traffic_occurrences = Occurrences\:\s+(?<Occurrences>\d[^\,]*)
EXTRACT-traffic_user_name = User\:\s(?<user>\w[^\,]+)
EXTRACT-traffic_local_pt = Local\sPort\:\s+(?<Local_Port>\d[^\,]*)
EXTRACT-traffic_remote_pt = Remote\sPort\:\s+(?<Remote_Port>\d[^\,]*)
EXTRACT-traffic_remote_name = Remote\s\Host\sName\:\s(?<Remote_Host_Name>.*?[^\,]*)
EXTRACT-traffic_remote_ip = Remote\sHost\sIP\:\s(?<Remote_Host_IP>\d[^\,]+)
EXTRACT-traffic_local_mac = Local\sHost\sMAC\:\s(?<Local_Host_MAC>\w[^\,]+)
EXTRACT-traffic_md5 = MD\-5\:\s(?<MD_5>.*?[^\,]*)
EXTRACT-traffic_sha256 = SHA\-256\:\s(?<SHA_256>.*?[^\,]*)
EXTRACT-traffic_app_name = Application\:\s(?<Application_Name>.*?[^\,]*)
EXTRACT-traffic_local_ip = Local\sHost\:\s+(?<Local_Host_IP>\d[^\,]+)
EXTRACT-traffic_remote_mac = Remote\sHost\sMAC\:\s(?<Remote_Host_MAC>\d[^\,]+)\,(?<Network_Protocol>\d[^\,]*)\,(?<Traffic_Direction>\w[^\,]+)
EXTRACT-traffic_vendor_action = Action\:\s(?<vendor_action>\w[^\,]+)
EXTRACT-traffic_rule_name = Rule\:\s(?<Rule_Name>\w[^\,]+)

Re: SEP 14.2 RU1 log format change

rubacker527 — Thu, 22 Aug 2019 13:04:54 GMT

Thanks for putting this together. It worked perfectly.

Re: SEP 14.2 RU1 log format change

dshpritz — Fri, 23 Aug 2019 14:43:25 GMT

You mean props.conf, correct?

Re: SEP 14.2 RU1 log format change

csperry_splunk — Fri, 23 Aug 2019 14:50:39 GMT

That is exactly what I meant. I just typed the wrong file name in.

Re: SEP 14.2 RU1 log format change

csperry_splunk — Fri, 23 Aug 2019 14:51:37 GMT

Fixed the bad file name---thanks

Re: SEP 14.2 RU1 log format change

GDustin — Wed, 30 Sep 2020 01:55:57 GMT

@csperry my guys just deployed 14.2RU1Mp1 any idea if that one is covered by your props method? Immediately my analysts told me; symantec:ep:security:file / [field_extraction_for_agt_security] went away; the only change I saw was "Local:" went to "Local Host IP:" in the raw logs, so I tried to rig some tests with | rex but it does not seem to fix all of it, but i am only just learning. If you have any insight or an update I'll buy you a cola and conf man. I'll see if I can put your props up on the SHC after hours tonight.

Re: SEP 14.2 RU1 log format change

GDustin — Tue, 27 Aug 2019 05:27:48 GMT

14.2RU1Mp1 (14.2.4814.1101).
https://support.symantec.com/us/en/article.TECH154475.html
Name Version/Build Release Date
(General Availability)
14.2.1.1 (14.2 RU1 MP1) 14.2.4814.1101 August 20, 2019

Re: SEP 14.2 RU1 log format change

GDustin — Wed, 30 Sep 2020 01:56:09 GMT

@csperry; False alarm, yes props method looks good and gets our dashboards populating again; Thanks.

`
symantec:ep:security:file : EXTRACT-security_local_ip
Inline Local\sHost\sIP:\s+(?\d[^\,]+) No owner Splunk_TA_symantec-ep Global

[“Original”] Local:\s+(?\d[^\,]+)
[“No Go”] Local Host IP:\s+(?\d[^\,]+)
[“Go”] Local\sHost\sIP:\s+(?\d[^\,]+)
`

Re: SEP 14.2 RU1 log format change

dsofoulis — Wed, 28 Aug 2019 05:49:27 GMT

Thanks, this worked for me

Re: SEP 14.2 RU1 log format change

jeremyhagand61 — Mon, 16 Sep 2019 06:10:56 GMT

This is a much better idea.