topic Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons in All Apps and Add-ons

Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

Anewec — Wed, 12 Apr 2017 21:34:38 GMT

I am trying to forward Windows and Linux logs directly from ArcSight logger to our Splunk environment. Since Arcsight converts logs to CEF format, I know the Splunk Add-ons for Windows and Linux will not work. From ArcSight, the logs are sent to Syslog server and then forwarded to Splunk.

What is the best way to get the Splunk Windows and Linux TA working with the ArcSight Win and Nix logs?

Thanks in advance!

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

gcusello — Thu, 13 Apr 2017 07:36:03 GMT

Hi,
I ingested logs sent by Arcsight by syslog without TAs, I only enabled a network input using the correct sourcetypes (WinEventLog:Security, ...).
The only problem is to separate logs if they are different, you have to override sourcetype by regex.
Beware, if you'receiving logs from Arcsight extractions that Archsight has a limit to 50,000 events.
Bye.
Giuseppe

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

Anewec — Fri, 14 Apr 2017 14:35:32 GMT

Thanks for your response Cusello! We actually have heavily customized pre-existing TA's that we are trying to leverage and the reason we want to be able to use the TAs.

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

gcusello — Fri, 14 Apr 2017 14:39:42 GMT

did you tried to use an intermediate Forwarder (or also one of the indexers) to receive syslogs from Arcsight (not using Splunk), pre parse logs writing them on a file and then ingest logs from the file using the existing TAs?
We used this workaround and runs!
Bye.
Giuseppe

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

Anewec — Fri, 14 Apr 2017 18:21:24 GMT

the data is currently coming to a heavy forwarder also the rsyslog server and then we are forwarding to Splunk. So pre-parse the logs on the HF, re-write to file on the HF before forwarding? Do you have any configurations u can share? Thanks!

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

gcusello — Sat, 15 Apr 2017 05:42:08 GMT

We pre parsed logs out of Splunk using a php script and then we ingested in the usual way (TAs).
Bye.
Giuseppe

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

maraman_splunk — Sat, 15 Apr 2017 13:57:39 GMT

I'm afraid there's no easy answer to this.
There are multiple way to do achieve it.
To go the Logger -> Splunk way , syslog and CEF , you probably better start with TA-cef , which will at least parse correctly CEF fields, then you'll have work to map the fields back to something that would mean something, probably focusing on making CIM compliant fields + some additionals for eventcode for exemple.
Of course, you can also do a simple regex to change the sourcetype at index time to avoid having only one sourcetype coming from logger.

Re: Ingesting ArcSight Windows and Linux Logs to Splunk with Splunk Add-ons

dflodstrom — Fri, 20 Oct 2017 21:27:24 GMT

If you use a connector appliance to manage your ArcSight connectors you can just add a new destination and point it at your Splunk server.

Add Destination > Create a new destination > Raw Syslog. Enter IP/Host, Port, Protocol (UDP), and select 'false' for metadata.

Enable a UDP syslog listener on the port you specified for your destination and have Splunk read the file.

This was my feedback in this similar question

I had this configuration working for a transition from arcsight to Splunk. You'll need a little bit of custom parsing on the Splunk side to get everything correct.