topic Re: FireEye Built in Dashboards Not Working in All Apps and Add-ons

FireEye Built in Dashboards Not Working

wbkendall — Mon, 17 Feb 2014 19:25:50 GMT

I installed the FireEye application and configured my WebMPS sensors to post alerts to Splunk per the instructions. I've reviewed the Splunkd log and can see where the events are making their way to Splunk.
The default FireEye dashboards have no results. I know the events are in the index because if I search for index=fe, my events show up.

Inspecting one of the dashboards shows the following query being run with no results:

search index=fe sourcetype="fe_xml" fe_appliance=* src_ip=* malware_name=* alert_id=* | rename severity AS fe_severity | eval fe_severity=replace(fe_severity,"\"","") | lookup severity fe_severity | rename severity AS SEVERITY | table alert_id fe_appliance src_ip dest_ip malware_name malware_stype SEVERITY

If I run this query manually, I get no results. If I start backing parameters of the query off, I get results at

index=fe sourcetype="fe_xml" fe_appliance=*

As soon as src_ip=* is tacked onto the search, results stop coming in.

I haven't modified any of the FireEye application files, and I've triple-checked that my sensors are sending data to Splunk in the Extended XML format. Does anyone have any ideas why this isn't working correctly? Thanks!

EDIT:

Looks like the src_ip isn't being extracted. I wonder if the event format changed? Here's an edited raw event:

<?xml version="1.0" encoding="utf-8"?><alerts appliance="ApplianceName" msg="extended" product="Web MPS" version="7.0.2.156588" xmlns="hxxp://www.fireeye.com/alert/2011/AlertSchema" xmlns:xsi="hxxp://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="hxxp://www.fireeye.com/alert/2011/AlertSchema FireEyeAlert.xsd"><alert id="405294" name="infection-match" severity="minr"><explanation analysis="content" protocol="tcp"><malware-detected><malware name="Local.Infection" sid="600104" stype="bot-command"/></malware-detected><cnc-services><cnc-service port="80" protocol="tcp"><address>141.101.117.X</address><channel>GET /efax_7132159010.doc hxxp/1.1::~~Accept: */*::~~User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET4.0C; .NET4.0E; ms-office; MSOffice 14)::~~Accept-Encoding: gzip, deflate::~~Host: slash.ma::~~Connection: Keep-Alive::~~Cookie: __cfduid=db3535ebf75b3f2976167c05678ac6e631392738961234::~~::~~</channel></cnc-service></cnc-services></explanation><src vlan="0"><ip>192.168.12.150</ip><host>host.domain.local</host><port>58268</port><mac>00:15:c7:00:00:00</mac></src><dst><ip>141.101.117.x</ip><mac>00:1b:54:f7:00:00</mac><port>80</port></dst><occurred>2014-02-18T18:00:16Z</occurred><interface label="A1" mode="tap">pether3</interface><alert-url>hxxps://sensor/event_stream/events_for_bot?ev_id=405294&amp;lms_iden=00:E0:81:C3:00:00</alert-url><action>notified</action></alert></alerts>

Re: FireEye Built in Dashboards Not Working

bosburn_splunk — Mon, 28 Sep 2020 15:54:40 GMT

I'd check to make sure the fe_appliance, src_ip, malware_name and alert_id fields are being extracted. I'd do a search for index=fe sourcetype="fe_xml" and see if the field shows up on the left hand side.

Brian

Re: FireEye Built in Dashboards Not Working

wbkendall — Tue, 18 Feb 2014 19:17:21 GMT

I couldn't post an event other than by editing the original post, but I now have one of the events in my post. I'm wondering if the format changed?

Re: FireEye Built in Dashboards Not Working

ccsfdave — Mon, 19 May 2014 20:01:41 GMT

I am having the same issue with src_ip not being extracted. Did you ever figure this one out?

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Mon, 28 Sep 2020 16:40:39 GMT

Thanks Brian. It appears that src_ip and malware_name are not being extracted properly. fe_appliance and alert_id are fine.

Re: FireEye Built in Dashboards Not Working

ccsfdave — Mon, 28 Sep 2020 16:40:44 GMT

Same with me, src_ip and malware_name (though name is) are not extracted. fe_appliance and alert_id are also fine

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Tue, 20 May 2014 20:25:22 GMT

Keep in mind that there are different "names" within the events. For example:

One "name" contained within the tag:

Another "name" contained within the tag:

The root cause of the field extraction issue may be located in the props.conf or the transforms.conf file?

Re: FireEye Built in Dashboards Not Working

ccsfdave — Mon, 28 Sep 2020 16:40:49 GMT

I am trying to extract the src_ip and malware_name fields and when I have Splunk write the regex in the "Interactive Field Extractor", it can't find the src_ip line probably because it is only pulling in the first 15 lines. Can I get this expanded? I could upload a screen shot if this is not clear...

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Wed, 21 May 2014 14:39:45 GMT

Try using the Universal fix before this one. I believe that should solve the problem no matter the version of FE OS. This will be left here just in case the other solution does not work for you.

Here is the fix for FireEye (pre-OS 7.1) appliances. This should work if your wMPS (NX) appliances are any version prior to 7.1. I will post a universal fix (that works for all appliances) if possible.

From the Splunk GUI:

App -> Manage Apps -> View Objects -> Search box: "EXTRACT-src-dst"

Click on the top hyperlink and replace the current contents with the following:

<src\svlan=\"\d+\"\><ip>(?<src_ip>[0-9.]+)</ip><host>(?<src_host>.*)</host><port>(?<src_port>\d+)</port><mac>(?<src_mac>[\w:]+)</mac></src><dst><ip>(?<dest_ip>[0-9.]+)</ip><mac>(?<dest_mac>[\w:]+)</mac><port>(?<dest_port>\d+)</port></dst>

Alternatively, you can also change this directly in the props.conf file:

Original:

EXTRACT-src-dst = \<src\svlan=\"\d+\"\>\<ip\>(?<src_ip>[0-9.]+)\</ip\>\<port\>(?<src_port>\d+)\</port\>\<mac\>(?<src_mac>[\w:]+)\</mac\>\</src\>\<dst\>\<ip\>(?<dest_ip>[0-9.]+)\</ip\>\<mac\>(?<dest_mac>[\w:]+)\</mac\>\<port\>(?<dest_port>\d+)\</port\>\</dst\>

Change to:

EXTRACT-src-dst = <src\svlan=\"\d+\"\><ip>(?<src_ip>[0-9.]+)</ip><host>(?<src_host>.*)</host><port>(?<src_port>\d+)</port><mac>(?<src_mac>[\w:]+)</mac></src><dst><ip>(?<dest_ip>[0-9.]+)</ip><mac>(?<dest_mac>[\w:]+)</mac><port>(?<dest_port>\d+)</port></dst>

Notice that the old regex did not take into account the tag which broke the parsing. Thanks.

Remember to restart splunkd after making this change: /opt/splunk/bin/splunk restart splunkd

Re: FireEye Built in Dashboards Not Working

ccsfdave — Wed, 21 May 2014 15:55:40 GMT

Thanks so much Tony, I am running 7.1.1 so if you get a chance for the universal fix, that would be great. In the mean time, I am going to see if the above works or can be manipulated to work for me.

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Wed, 21 May 2014 16:48:57 GMT

A Universal Solution (Works for any version of FireEye OS) should be below:

FireEye 7.1 and later uses "Pretty print" for the XML. While this increases the readability for humans, it does break the Splunk parsing in the current app. We can account for the extra carriage returns, line feeds, and tabs using the following: ([\r\n]*)\s*

There appear to be two Extraction parameters that need a slight modification to make all of the dashboards function again: EXTRACT-src-dst and EXTRACT-malwaredetected

From the Splunk GUI:

App -> Manage Apps -> View Objects -> Search box: "EXTRACT-src-dst"

Click on the top hyperlink and replace the current contents with the following:

<src\svlan=\"\d+\"\>([\r\n]*)\s*<ip>(?<src_ip>[0-9.]+)</ip>([\r\n]*)\s*<host>(?<src_host>.*)</host>([\r\n]*)\s*<port>(?<src_port>\d+)</port>([\r\n]*)\s*<mac>(?<src_mac>[\w:]+)</mac>([\r\n]*)\s*</src>([\r\n]*)\s*<dst>([\r\n]*)\s*<ip>(?<dest_ip>[0-9.]+)</ip>([\r\n]*)\s*<mac>(?<dest_mac>[\w:]+)</mac>([\r\n]*)\s*<port>(?<dest_port>\d+)</port>([\r\n]*)\s*</dst>

Notice that the old regex did not take into account the <host> tag which also broke the parsing. This new one does.

Now change EXTRACT-malwaredetected...

From the Splunk GUI:

App -> Manage Apps -> View Objects -> Search box: "EXTRACT-malwaredetected"

Click on the top hyperlink and replace the current contents with the following:

<malware-detected>([\r\n]*)\s*<malware\sname=\"(?<malware_name>.*)\"\s+sid=\"(?<malware_sid>.*)\"\s+stype=\"(?<malware_stype>.*)\"/> in _raw

Remember to restart splunkd after making this change: /opt/splunk/bin/splunk restart splunkd

Let me know if this works for you. Thanks.

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Wed, 21 May 2014 22:18:06 GMT

No problem Dave. Let me know if the Universal fix works for you. I tested it on FireEye OS 6.2 and 7.1 and it seems to be working.

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Thu, 22 May 2014 13:32:41 GMT

So others don't need to to make this change... We patched the Splunk for FireEye app to version 2.0.8 (making the change above) and pushed it to the Splunk apps site. I also created a howto for installation on Splunk v6 as a stopgap until the new app is completed. See the instructions below:

http://securitysynapse.com/2014/05/stopgap-splunk-for-fireeye-v2-app.html

Let us know if you have any issues in the mean time,
-Tony

Re: FireEye Built in Dashboards Not Working

srioux — Wed, 16 Jul 2014 14:19:35 GMT

Side note, some of the events may or may not have a "host" field in them, which provides some inconsistencies with how the src-dst extractions work. My workaround just put another field extraction in place to catch the ones that didn't have a host field:

<src\svlan=\"\d+\"\>([\r\n]*)\s*<ip>(?<src_ip>[0-9.]+)</ip>([\r\n]*)\s*<port>(?<src_port>\d+)</port>([\r\n]*)\s*<mac>(?<src_mac>[\w:]+)</mac>([\r\n]*)\s*</src>([\r\n]*)\s*<dst>([\r\n]*)\s*<ip>(?<dest_ip>[0-9.]+)</ip>([\r\n]*)\s*<mac>(?<dest_mac>[\w:]+)</mac>([\r\n]*)\s*<port>(?<dest_port>\d+)</port>([\r\n]*)\s*</dst>

Re: FireEye Built in Dashboards Not Working

TonyLeeVT — Tue, 26 Aug 2014 19:14:55 GMT

Thanks for hanging in there. Please try out our new app designed to work with Splunk v6. We increased the number of FireEye appliances we support and drastically increased the number of protocols you can use to send data to Splunk. Download it here: http://apps.splunk.com/app/1845/. It is still in BETA, but we are accepting user feedback at: Tony.Lee -at- FireEye.com. Thanks!