What are the advantages and disadvantages of using Splunk as an alternative log management system for syslog-ng or ryslog log management?
Okay, here's what I know about that question. In our shop, that's not an either-or. The other two manage the logs on the individual systems, then those logs are loaded into splunk for analysis, tracking, permanent retention and so on.
I know nothing about the analytics abilities of rsyslog or syslog-ng, but I suspect if they exist at all, they are rudimentary, so they do not substitute for splunk in any way.
I have never heard anyone claim that splunk did anything independently regarding CREATING or MANAGING logs on individual boxes, so splunk does not substitute for rsyslog or syslogng in any way.
There is a zone in the middle where there are choices -- deciding what to log on the box at all, which must then be transmitted to splunk, and then deciding what events to index in splunk vs what events to send to the null queue -- where there are architectural decisions to be made. But that's a question of which application is throwing stuff away, how much transmission to do, and so on, not a question of
If I'm interpreting your question correctly, you're asking why use Splunk instead of just a syslog server for managing logs... the problem is these serve different purposes, and in a well architected environment, you will likely have both Splunk AND syslog servers... Specifically in regards to Splunk and Syslog, George Starcher's has the authoritative blog on this topic.
In a log management setup, you first need to collect logs. Syslog-ng and rsyslog do this very well for logs sent over the syslog protocol and you would be a fool to not standardize on the one that has the most knowledge base at your organization. But syslog is only a single source of data. There is a lot of other places where interesting and useful data flows through your organization, possibly including but not limited to Windows Event Logs, Application Log files, Database entries, and Packet Data... the question is then how to gather that data if applicable for your goals, and that would be a fit for Splunk and other solutions.
Now that you have paid the cost to gather logs centrally, you probably want to do something with it... be it searching, correlation, trending, dashboarding, alerting, and reacting to what is happening in your environment, across all sources of data. You also may want to let people have access to different sets of the data that you have gathered. Finally, you'll need to understand your retention requirements... How much of different log types, how long do you keep logs searchable, and what do you do with them when not... maybe you need to ensure you have some of your logs distributed to multiple sites in case of emergencies.
These are some of the things that you'd use Splunk to accomplish. The biggest complaint I hear about Splunk is that it's licensed software, and therefore comes with costs... but you'll need to figure out for your organization if the expertise in the half dozen or so other projects that you'd need to setup to take it's place to deliver the same functionality is worth it or not.
@acharlieh - yeah, I guess everyone who has a lawn needs to decide whether then need to go back to a push mower, too, instead of paying for the gas or electricity... 😉
Me, I'd figure for any medium to large shop, the costs of best-in-class SEIM would be a slam dunk... unless you have an installed base of something else that fully meets your needs.