I am planning to start learning about Splunk. I wanted to know the difference between Splunk and HP Arcsight. I have googled for it, but couldnt get convincing answers. Hence, thought of posting here.
To add on to your Q & A, I have worked as a ArcSight engineer for two years and have been working with Splunk for the last two years.
One of the biggest down falls with ArcSight is getting timely technical support. Secondly the content that is out of the box, if not turned off, can cause issues with the ESM in a very short period of time because it's not data specific. Also HP also does not provide a free ready-to-use downloadable version of the ESM and connectors where you can began learning more about building rules,use cases,dashboards,active list...etc.
Splunk does provide this as well as apps along with developer license so you can build upon your skill and get hands on experience.
I was an ArcSight consultant in the past so I'll try to help here.
First of all, comparing ArcSight or any other SIEM with Splunk is like comparing apples and pears. Splunk is not a SIEM. Splunk is a data analysis and collection tool. Any type of data and not just security related one. Forget about mapping your ip address field in your logs to the Source.IP field, just focus on ingesting and analysing your data. No matter what format it is.
Now, with regards to your questions:
1 - Does Splunk do same job as Arcsight used in Security Operation center.
Splunk by itself won't provide built-in rules, alerts, etc, which is something you will find in ArcSight. Splunk Enterprise can collect and store all your data and if you use the existing add-ons you can automatically "normalise" your sources in the same way as ArcSight would do, but with the added value that you can modify this existing add-ons to match your requirements without having to write a FlexConnector from scratch.
If you are looking for the SOC aspects of Splunk you should probably deploy Enterprise Security or alternatively, if your requirements are not very complex, you can even build your own SOC app. I've taken both approaches depending on what the customer was trying to achieve.
2 - Can splunk be used to monitor/feed threat intelligence feeds and create threat intelligence reports.
Yes. See this and this. Enterprise Security can also help (see this). The Getwatchlist app can be extremely useful too.
3 - I downloaded free version of splunk..can I use it as SIEM tool for learning purpose
It depends. You will be able to collect up to 500MB of data per day using the free version but you can install as many universal forwarders as you like and configure them to collect your logs. If you use the existing built-in apps for your sources then you'll find lots of interesting dashboards and visualisations or interesting searches.
For example, let's say you have some Cisco ACS devices you want to monitor. You just need to configure those devices to send the Syslog traffic to the collectors (this could be a Splunk Enterprise instance or a Universal Forwarder listening on UDP 514 or whatever you use, a Syslog-NG or Rsyslog server that writes that into disk allowing the Universal Forwarder to read it locally, etc. See this post), install the Cisco Add-on for ACS, configure Splunk to read the logs, collect your logs, send them to the central server, search, create rules and visualisations, done.
4 - Is splunk enterprise security tool and free version same ie., can i use it to monitor threat logs.
ES is just an app that runs on top of a fully licensed Splunk Enterprise and is not free. Most of the apps are, but ES, VMWare, UBA, PCI and a few others are not.
It will normally cost around 25% of what your full Splunk Enterprise license costs if I remember correctly. Obviously the more data you ingest and the bigger the license is, the cheaper the cost per GB. You will need to contact Splunk or a reseller in order to buy ES.
Hope that helps.