Parsing Forcepoint CASB CEF logs in Splunk

geoffmoraes — Fri, 18 Sep 2020 07:37:26 GMT

I need some help with parsing Forcepoint CASB CEF logs in Splunk. The data does not seem to parse the epoch time stamps and all comes in as one event. I need to break these up into individual events and also parse the epoch time stamp in the format "%Y-%m-%d %H:%M:%S" on ingestion into splunk.

SAMPLE DATA

CEF:0|Forcepoint CASB|Cloud Service Monitoring|1.0|63575192763|Activity|0|act=Monitor app= cat=Normal Activity cs1= destinationServiceName=Office365 deviceExternalId= deviceFacility=true deviceProcessName=loadBalancers dhost= dpriv=User dst=0.0.0.0 duser=14dd43c6-a792-4c07-a33b-c5e561a129de dvc=10.1.2.12 dvchost=somedomain end=1600408516000 externalId=0 fsize=-1 msg=//United States/Unknown outcome=Success proto= reason=modify request= requestClientApplication=Unknown/Unknown/"" rt=1600408516000 sourceServiceName=Unmanaged src=someIP start=1600408516000 suser= cs2= cs3= cs5=false cs6= dproc=Unknown flexString1=mc_rg-int-qa-eus02_aks-int01-qa-eus02_eastus2,kubernetes-internal cs4=14dd43c6-a792-4c07-a33b-c5e561a129de flexString2= AD.ThreatRadarCategory= AD.TORNetworks= AD.MaliciousIPs= AD.AnonymousProxies= AD.IPChain=someIP AD.IPOrigin=External AD.samAccountName=14dd43c6-a792-4c07-a33b-c5e561a129de CEF:0|Forcepoint CASB|Cloud Service Monitoring|1.0|63575216734|Activity|0|act=Monitor app= cat=Normal Activity cs1= destinationServiceName=Office365 deviceExternalId= deviceFacility=false deviceProcessName= dhost= dpriv=User dst=0.0.0.0 duser=someuser_849fcb2777e9@somedomain.onmicrosoft.com dvc=10.1.2.12 dvchost=somedomain end=1600406172000 externalId=0 fsize=-1 msg=//United States/Unknown outcome=Success proto= reason=login request= requestClientApplication=Unknown/Unknown/"" rt=1600406172000 sourceServiceName=Unmanaged src=someIP start=1600406172000 suser= cs2= cs3= cs5=false cs6= dproc=Unknown flexString1= cs4=someaccount9@somedomain.onmicrosoft.com flexString2= AD.ThreatRadarCategory= AD.TORNetworks= AD.MaliciousIPs= AD.AnonymousProxies= AD.IPChain=someIP AD.IPOrigin=Internal AD.samAccountName=someaccount9@somedomain.onmicrosoft.com CEF:0|Forcepoint CASB|Cloud Service Monitoring|1.0|63575216736|Activity|0|act=Monitor app= cat=Normal Activity cs1= destinationServiceName=Office365 deviceExternalId= deviceFacility=true deviceProcessName= dhost= dpriv=User dst=0.0.0.0 duser=someaccount@somedomain.onmicrosoft.com dvc=10.1.2.12 dvchost=somedomain end=1600405713000 externalId=0 fsize=-1 msg=//United States/Unknown outcome=Success proto= reason=login request= requestClientApplication=Unknown/Unknown/"" rt=1600405713000 sourceServiceName=Unmanaged src=someIP start=1600405713000 suser= cs2= cs3= cs5=false cs6= dproc=Unknown flexString1= cs4=someaccount@somedomain.onmicrosoft.com flexString2= AD.ThreatRadarCategory= AD.TORNetworks= AD.MaliciousIPs= AD.AnonymousProxies= AD.IPChain=someIP AD.IPOrigin=External AD.samAccountName=someaccount@somedomain.onmicrosoft.com CEF:0|Forcepoint CASB|Cloud Service Monitoring|1.0|63575216738|Activity|0|act=Monitor app= cat=Normal Activity cs1= destinationServiceName=Office365 deviceExternalId= deviceFacility=false deviceProcessName= dhost= dpriv=User dst=0.0.0.0 duser=someaccount@somedomain.com dvc=10.1.2.12 dvchost=somedomain end=1600405674000 externalId=0 fsize=-1 msg=/1225/United States/Unknown outcome=Success proto= reason=login request= requestClientApplication=Unknown/Unknown/"" rt=1600405674000 sourceServiceName=Unmanaged src=someIP start=1600405674000 suser= cs2= cs3= cs5=false cs6= dproc=Unknown flexString1= cs4=someaccount flexString2= AD.ThreatRadarCategory= AD.TORNetworks= AD.MaliciousIPs= AD.AnonymousProxies= AD.IPChain=someIP AD.IPOrigin=Internal AD.samAccountName=someaccount CEF:0|Forcepoint CASB|Cloud Service Monitoring|1.0|63575216735|Activity|0|act=Monitor app= cat=Normal Activity cs1= destinationServiceName=Office365 deviceExternalId= deviceFacility=false deviceProcessName= dhost= dpriv=User dst=0.0.0.0 duser=someaccount9@somedomain.onmicrosoft.com dvc=10.1.2.12 dvchost=somedomain end=1600406165000 externalId=0 fsize=-1 msg=//United States/Unknown outcome=Success proto= reason=login request= requestClientApplication=Unknown/Unknown/"" rt=1600406165000 sourceServiceName=Unmanaged src=someIP start=1600406165000 suser= cs2= cs3= cs5=false cs6= dproc=Unknown flexString1= cs4=someaccount@somedomain.onmicrosoft.com flexString2= AD.ThreatRadarCategory= AD.TORNetworks= AD.MaliciousIPs= AD.AnonymousProxies= AD.IPChain=someIP AD.IPOrigin=Internal AD.samAccountName=someaccount@somedomain.onmicrosoft.com

Assume that the sourcetype is called "cefevents" for this example.

PROPS.CONF

[cefevents] LINE_BREAKER = CEF:0.+[\r\n]? MAX_TIMESTAMP_LOOKAHEAD = 600 NO_BINARY_CHECK = true KV_MODE = none SHOULD_LINEMERGE = false TIME_FORMAT = %s%Q TIME_PREFIX = \s(start|end|rt)\=

The regex looks ok on regex101 https://regex101.com/r/hf7ZJs/1 but doesn't work on this data.

I have also attempted using the props.conf from https://splunkbase.splunk.com/app/487/ but that does not help either.

Re: Parsing Forcepoint CASB CEF logs in Splunk

thambisetty — Fri, 18 Sep 2020 12:25:00 GMT

TIME_PREFIX - you can't set multiple prefixes. set any of them. I have used end=( you can change it)

TIME_FORMAT - you can format the way you want at the time of search. we can't enforce to show time format for each source type. TIME_FORMAT will tell splunk in which format time is there in event. so that Spunk understands time correctly.

[ __auto__learned__ ] SHOULD_LINEMERGE=false LINE_BREAKER=([\r\n]+) NO_BINARY_CHECK=true TIME_PREFIX=end\=

Re: Parsing Forcepoint CASB CEF logs in Splunk

geoffmoraes — Wed, 23 Sep 2020 12:02:06 GMT

@thambisettyIf I take a sample cef file and upload it to splunk, these values work as expected.

However, if I create a custom deployment app on a Heavy Forwarder with these values in props.conf, it has no effect on the incoming logs. The logs have no line breaks and the time stamp is that of the log file drop on the server with the UF.

Re: Parsing Forcepoint CASB CEF logs in Splunk

thambisetty — Wed, 23 Sep 2020 12:12:06 GMT

Did you set sourcetype? you should should use sourcetype that you have mentioned in inputs.conf in heavy forwarder props.conf.

Re: Parsing Forcepoint CASB CEF logs in Splunk

geoffmoraes — Wed, 23 Sep 2020 12:20:46 GMT

@thambisetty Yes, the sourcetype in props.conf matches what is set in inputs.conf for this custom Forcepoint CASB app.

Re: Parsing Forcepoint CASB CEF logs in Splunk

geoffmoraes — Thu, 24 Sep 2020 08:43:29 GMT

This has been resolved.

I had the props.conf under

$SPLUNKHOME\etc\deployment-apps\<app_name>\local\

I removed that file and instead added its contents to the existing props.conf under

$SPLUNKHOME\etc\apps\search\local\

and the line-breaking and time-stamp parsing work as expected.

topic Re: Parsing Forcepoint CASB CEF logs in Splunk in Getting Data In

Parsing Forcepoint CASB CEF logs in Splunk

Re: Parsing Forcepoint CASB CEF logs in Splunk

Re: Parsing Forcepoint CASB CEF logs in Splunk

Re: Parsing Forcepoint CASB CEF logs in Splunk

Re: Parsing Forcepoint CASB CEF logs in Splunk

Re: Parsing Forcepoint CASB CEF logs in Splunk