All Apps and Add-ons

Why are CEF Events not all extracting correctly, shifting key-value pairs for pipe-delimited fields?

jravida
Communicator

Hi Folks,

I'm ingesting CEF events coming in from ArcSight and getting pushed over SYSLOG to a file, which I have Splunk read. I use this method for a few different types of sources, but with one of them the parsing isn't happening correctly.

It's as if it ignores the cef_vendor field value and puts the cef_product value in there instead. So instead of saying "cef_vendor = McAfee" (I can see within the event that McAfee is occurring after the first pipe in "CEF:0|McAfee|ePolicyOrchestrator|..." ), it will say ePolicyOrchestrator, which is actually the cef_product. And from there other CEF "pipe-delimited" fields are populating wrong. After the CEF pipe-delim fields everything seems to be parsing correctly (the extension fields)

I don't understand why this is parsing incorrectly. All my other feeds come in the same manner, with arcsight-->syslog--->splunk file/folder reader. I shoulod also note that my KV_MODE is set to NONE for this input.

Thanks in advance!

0 Karma

grantsales
Engager

Did you get this working?

I found some regex errors with "+" meaning one or more. Change these out to "*" and you'll get cef parsing even if the field is empty.

0 Karma

IgorB
Path Finder

CEF standard says that the version field is mandatory, so "+"es are not an error.

Header Information

CEF uses syslog as a transport mechanism. It uses the following format, comprised of a syslog prefix, a header and an extension, as shown below:

Jan 18 11:07:53 host CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|Name|Severity|[Extension]

The CEF:Version portion of the message is a mandatory header. The remainder of the message is formatted using fields delimited by a pipe ("|") character

...

ppablo
Retired

Hi @jravida

You tagged "CEF (Common Event Format) Extraction Utilities" App to your post, but I didn't see you mention it anywhere in the content. I just wanted to check, but have you actually been using that app to parse your CEF Arcsight logs? Sometimes people tag applications in their question on accident because it's the first tag that populates on a list so just wanted to make sure. If not, then that app might help with your field extraction issue. Here's the link to the app's page https://apps.splunk.com/app/487/

jravida
Communicator

I am in fact using the CEF extraction util

0 Karma

IgorB
Path Finder

Happy to hear that.

Now back to your question - could you please provide data examples so I'd be able to try figuring out what causes parsing problems with McAfee events?

0 Karma

jravida
Communicator

This one doesn't work:

Sep 30 20:27:13 fqdn.mynetwork.com CEF: 0|McAfee|ePolicy Orchestrator|1.0|||Unknown|

But this one extracts and parses fine.
Sep 30 20:27:13 fqdn.mynetwork.com CEF: 0|McAfee|ePolicy Orchestrator|1.0|10196|Access Logged - start|Unknown|

0 Karma

IgorB
Path Finder

I'm unable to reproduce the problem with the data in your example

screenshot

BTW, your data has space between "CEF:" and "0". Is this way in your real data?

0 Karma

IgorB
Path Finder

jravida Could you please send me data examples for both correctly and incorrectly parsed events?

0 Karma

IgorB
Path Finder

LukeMurphey - app's documentation says that it's for sending Splunk data to ArcSight, and not the other way around...

[

The Splunk App for CEF enables you to aggregate and augment Splunk Enterprise events, transforming them into the Common Event Format (CEF), an open log management standard.

The Splunk App for CEF produces output that consists of your search results reformatted into the Common Event Format. You can then use the CEF output for further processing in compatible applications such as HP ArcSight.]1

0 Karma

LukeMurphey
Champion

In case you are interested, there is an officially supported solution for getting CEF data into Splunk from ArcSight. See https://apps.splunk.com/app/1847/.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...