Splunk App for Enterprise Security: How to parse key value pairs for Incapsula WAF and API output?

[dcroteau]

Hello,

We are using an Incapsula WAF and using a curl script to pull out the timestamps and security events. How do we parse this?

[dcroteau]

Sample Data:

[AccountID=11111] 
[AccountName=BlaBla] 
[SiteId=22XX46] 
[SiteName=www.blabla.com] 
[EventID=188000790104000832] 
[EventTimestamp=1429486146594] 
[EventType=ThreatAlert] 
[ClientIP=255.255.255.222:10] 
[ClientApp=Genieo:1]
---- VISITS ---- 
---- VISIT ----
[VisitID=1880000000104000832][StartTime=2015/04/18 18:12:12 +0000] [Timestamp=1429380732975] [ClientApplication=Genieo][ClientType=Crawler] 
[UserAgent=Mozilla/1.1 (compatible; Genieo/1.0 http://www.xxxx.com/webfilter.html)] [SupportsCookies=COOKIES_NOT_DETERMINED] 
[SupportsJavaScript=JS_NOT_DETERMINED] 
[ClientIP=255.255.255.233] 
[Country=France] 
[ServedVia=Paris, France] 
[NumberOfHitsOnVisit=1][NumberOfPageViewsOnVisit=0] 
[EntryReferer=] 
[EntryPage=www.blabla.com/robots.txt]   
Request [URL=/robots.txt][ResponseCode=0][RequestResult=REQ_BLOCKED_SECURITY]       
[NumRequests=1]         
[RequestsIndexOnVisit=1]        
[QueryString=]      
[PostData=]         
[Referer=]      
[IncidentID=1880000000104000832-489984751201682682]      
Attack Info:        
[Rid=4][RuleName=Bad Bots]      
[ActionTaken=Request blocked]                               -- Attack Vector:           
        [AttemptedOn=URL]           
        [ThreatPattern=www.blabla.com/robots.txt]                       [AttackInternalCode=200.0]

[dcroteau · Accepted solution]

Assuming that your chosen sourcetype is Event_Incapsula in your incapsula Apps props.conf.

[Events_Incapsula]
SHOULD_LINEMERGE = true
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE = ^\[VisitID\=
MAX_TIMESTAMP_LOOKAHEAD=100
TIME_PREFIX=(StartTime)\=
EXTRACT-RuleName = RuleName\=(?([^]]*))
*EXTRACT-ActionTaken = ActionTaken\=(?([^]]*))*

All the key value pairs should be created.

[dcroteau]

event format:

[VisitID=266000350049708189][StartTime=2015/04/18 18:44:43 +0000] [Timestamp=1429382683277] [ClientApplication=Known Vulnerability Scanner][ClientType=Worm] [UserAgent=Mozilla/4.0 (compatible; MSIE 77; Windows NT 7.1; SV1; .NET CLR 2.0.50727)] [SupportsCookies=COOKIES_PENDING] [SupportsJavaScript=JS_NOT_DETERMINED] [ClientIP=255.234.255.255] [Country=United States] [ServedVia=Hong Kong] [NumberOfHitsOnVisit=1][NumberOfPageViewsOnVisit=1] [EntryReferer=http://www.urlblabla.com] [EntryPage=www.urlblabla.com/] -- Request [URL=www.urlblabla.com/][ResponseCode=0][RequestResult=REQ_BLOCKED_SECURITY] [NumRequests=1] [RequestsIndexOnVisit=1] [QueryString=] [PostData=] [Referer=] [IncidentID=2666666350049708189-144998435232809528] -- Attack Info: [Rid=4][RuleName=Bad Bots] [ActionTaken=Request blocked] -- Attack Vector: [AttemptedOn=URL] [ThreatPattern=www.urlblabla.com/] [AttackInternalCode=200.0] ================================================== max-ts: 1429382683277