According to the SPLUNK App for CEF documentation:
3) Use the guided search wizard included in the Splunk App for CEF to define what the output will look like in CEF by selecting a data model, mapping data model attributes to fields where necessary (a good amount of this work will be done automatically), creating any new static fields you need, and defining the name of the syslog receiver that will receive the data.
You mention that the SPLUNK app for CEF provides a continuous export of the data from SPLUNK which sounds great, but the question I have on this is "Do you have to map every event one by one first or is there some way to just get a full export of the SPLUNK data all at once?"
Thanks in advance.
It seeams that the application is bad and the document doesn't contain lot of information regarding the configuration of the data model
Not very encouraging then, is it?
I'm surprised we cannot get an answer for this SPLUNK app from the SPLUNK team...
I suspect the problem is that people don't understand what this means: "Do you have to map every event one by one first or is there some way to just get a full export of the SPLUNK data all at once?"
Could you clarify? The app doesn't export a single event but all events within the selected data-model. By event, do you mean data-model?
By event I mean every event with an event ID. From the description above, it looks like data model is translated into events?
In any case though, it looks like you create a "search description" within the SPLUNK app for CEF and using that search description SPLUNK Enterprise writes the data in CEF to the Syslog receiver you specify.
What about other receivers such as Sourcefire, Tripwire, Symantec, Mcafee, etc.?
SPLUNK app for CEF looks like it may work for Syslog data but I don't know about other different log types.
Also, I have info from others out in the field regarding the SPLUNK to ArcSight integration:
"Two things it doesn't seem to address is the timing issues and health monitoring. It's aim is to fix the formatting issues so you can use a standard ArcSight connector in a standard configuration but it doesn't seem to address any other the other issues with relaying events through Splunk. Two of the biggest problems with relaying your events through Splunk is the timing issues it creates with late events and the lack of device/health monitoring that you would normally have if you were getting it straight from a An ArcSight connector. I can elaborate further if needed but it is my understanding that the Splunk CEF app only addresses formatting it doesn't address delivery, event time or health monitoring issues which plague that configuration."
Hi, thanks -- that helps me understand the question.
1) The app works by modeling data that has been received into Splunk and then emitting a CEF formatted stream. Its functionality hasn't got anything to do with event IDs, at least as we understand them.
2) If your receiver doesn't want CEF over syslog, then you probably don't need this app, and can use Splunk's native syslog output or outputcsv or pull from one of our APIs instead. You may still find the app handy as an example of how you can manipulate data for such an output though.
3) I agree that the app is not intended or designed for that purpose. A better way to handle such a use case is to do it in Splunk directly and pass the results to the next system down the pipeline.
I think that the documentation is not enough detailled.
1) Splunk App for CEF (SACEF)
Normally the goal of this application, if I'm correct, will be to translate data from incoming data received by Splunk, using a Data Modal, wich is a query in the Splunk database, that select event we want to translate into CEF format. This CEF format is then sent to a syslog server or an ArcSight Connector.
The main problem I think here, is that we need to create a Data Model for all logs to send via CEF format.
I mean, we have to indicate to SACEF all datasource to translate into the CEF format. This mean that we need to modify the Data Model for each new sourcetype we want to translate in to CEF.
2) Receiving CEF into Splunk
Now, I'll be surprise how to configure Splunk to receive and understand logs from CEF flow, such as from ArcSight Connector or ArcSight Logger?
I found this Splunk App ==> https://apps.splunk.com/app/487/
But again a think the document is not enough detailled ... 😕 Where to install it? (on the Indexer? on the head search? on the Universal Forwarder?
Lot of application or network security devices work with CEF syslog flow. I'm surprise that Splunk doesn't have a document that describe how to configure it...
From your comment above: "I mean, we have to indicate to SACEF all data sources to translate into the CEF format. This mean that we need to modify the Data Model for each new sourcetype we want to translate in to CEF."
Even if you can somehow get these data models configured correctly you still have timing and delivery issues as well as health monitoring issues...
For security monitoring you cannot sacrifice accurate event times or correct delivery of events. You will have loss of data integrity which is unacceptable when reporting security events.
Is there any way that SPLUNK can look at some of these data integrity issues when trying to integrate with ArcSight?
Does the SPLUNK team agree or disagree that there are data integrity issues with the SPLUNK ---> ArcSight integration?
Any feedback is appreciated.. Thanks
Yes, I agree.
I read a doc that explain how to use a heavy forwarder for such thing... but it doesn't support CEF...
I don't know if a Splunk Professional Service read the forum to explain how to deal with this difficulties...
Thank you, I've updated the docs to try and clarify the point of confusion about what App for CEF does.
It's not accurate to say that you need to alter your data model for every sourcetype that you want to send... rather you might add a transform to that sourcetype. Much of the time, you'll find that this is already done because the Add-on or App that helped gather the data is Common Information Model compliant. An example might help: