All Apps and Add-ons

How can I fix Symantec Endpoint Protection (SEP) field extractions (again)?

gf13579
Communicator

Symantec have changed their logging format for the risk log (AV alerts) again. How can I update my search head to parse the fields correctly?

0 Karma
1 Solution

gf13579
Communicator

December 2019 update

Here's a working field_extraction_for_agt_risk transform in the symantec:ep:risk:file sourcetype:

(?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?<Computer_Name>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,\s*(?:Event\sInsert\sTime:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End\sTime:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?<Last_Update_Time>[[sep_file_field]]))?,\s*(?:Domain\sName:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group Name:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server\sName:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?<user>[[sep_file_field]]),\s*(?:Source\sComputer\sName:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sComputer\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?<Reason_For_White_Listing>[[sep_file_field]]),\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:Application\sname:\s(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P<Application_Version>.*))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?(?:,\s*Category\sset:\s*(?<Category_Set>[[sep_file_field]]),\s*Category\stype:\s*(?<Category_Type>[[sep_file_field]]))?,?\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?

For reference, here were the field name changes:

Inserted -> Event Insert Time
End -> End Time
Domain -> Domain Name
Group -> Group Name
Server -> Server Name
Source computer -> Source Computer Name
Source IP -> Source Computer IP

View solution in original post

JustinV
Engager

This is awesome, thanks for posting this!!! Did you do any extractions for the other symantec sourcetypes? Was hoping someone has so we don't have to go through this crazy regex 🙂

0 Karma

gf13579
Communicator

Hi Justin. Glad it helped.

I'm aware the other sourcetypes are also broken, I just didn't have time (or enthusiasm) enough to fix them. Comparing what's misssing/renamed is a pain.

Let me know which of the other sourcetypes is the most important and I'll try to give it a go over coffee.

0 Karma

JustinV
Engager

Thanks, we had to re-write these for the 14.2 RU1 and was hoping someone had done them for 14.2 RU2. The next ones we will work on are symantec:ep:security:file & symantec:ep:behavior:file. If we figure them out, I'll also paste them on this thread so others can have them.

0 Karma

JustinV
Engager

We ended up using the regex from this post where csperry broke out the fields to individual regexes. That got us running and should help keep better compatibility when Symantec adds new fields. This doesn't work for every sourcetype but does it for the main ones needed to do risk analysis.

https://answers.splunk.com/answers/745774/sep-142-ru1-log-format-change.html

0 Karma

deepamshah
Explorer

Hello,

Just wondering if anyone found the fix for 14.2 RU2 and 14.2 RU2 MP1 ?

Thanks in advance

0 Karma

rriegert
New Member

I second deepamshah's comment, looking for 14.2 RU2 MP1 field parsers also.

0 Karma

gf13579
Communicator

December 2019 update

Here's a working field_extraction_for_agt_risk transform in the symantec:ep:risk:file sourcetype:

(?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?<Computer_Name>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,\s*(?:Event\sInsert\sTime:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End\sTime:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?<Last_Update_Time>[[sep_file_field]]))?,\s*(?:Domain\sName:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group Name:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server\sName:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?<user>[[sep_file_field]]),\s*(?:Source\sComputer\sName:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sComputer\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?<Reason_For_White_Listing>[[sep_file_field]]),\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:Application\sname:\s(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P<Application_Version>.*))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?(?:,\s*Category\sset:\s*(?<Category_Set>[[sep_file_field]]),\s*Category\stype:\s*(?<Category_Type>[[sep_file_field]]))?,?\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?

For reference, here were the field name changes:

Inserted -> Event Insert Time
End -> End Time
Domain -> Domain Name
Group -> Group Name
Server -> Server Name
Source computer -> Source Computer Name
Source IP -> Source Computer IP