Splunk Tech Talks
Deep-dives for technical practitioners.

Splunk Connect for Syslog: Turnkey and Scalable GDI

Splunk Employee
Splunk Employee

View our Tech Talk: Platform Edition, Splunk Connect for Syslog: Turnkey and Scalable GDI 

Splunk Connect for Syslog is a containerized Syslog-ng server with a configuration framework designed to simplify getting syslog data into Splunk Enterprise and Splunk Cloud. This approach provides an agnostic solution allowing administrators to deploy using the container runtime environment of their choice. Additionally, skilled deployment engineers with syslog server experience can utilize the source and configurations directly to craft their own custom solutions more easily and more consistently than starting from scratch.

Tune in to:

  • Learn how easy it is to both onboard sources utilizing the default syslog port or sources that are on a custom port.
  • Learn all about a new Repeatable, Concise, Scalable, and Prescriptive Splunk solution for syslog GDI.
  • Understand how removal of the UF reduces configuration and management effort.
  • See how easy turnkey deployment via the SC4S container architecture is.

Tech Talk discussions remain open for two weeks following the live Tech Talk event. Have more questions?View our syslog tag in community for more.

Splunk Employee
Splunk Employee

Here is all the great Q&A from the live Tech Talk.


Q: Can we do TLS?

A: Yes -- you can do TLS on both inbound and outbound.


Q: Does sc4s has better error handling? 

A: Yes, it has a disk buffer for connection errors to HEC, and several error-handling routines in each "log path" (filter) for timestamp anomalies, etc.


Q: Does sc4s support both RFC formats?

A: Yes, both RFC 3164 and 5424. As well as _many_ which don't send in either format.


Q: Can you suppress syslog-ng parsing and process raw messages?

A: Yes, but that does require more resources as syslog-ng does not save the raw message by default. But if this is necessary, it is available but is typically only used for debugging situations. There are other cases where you need the raw message forwarded on to another (perhaps syslog) receiver, that is possible too.

Q: Side by side?

A: It depends on the format of the incoming message. The proper parser (3164 or 5424) is invoked depending on the message.


Q: Is syslog-ng exposed so I can forward out to other syslog/app repositories?

A: You can use the BYOE option to implement custom use cases like this.


Q: How friendly is running syslog-ng on a RHEL distro where rsyslog is the native syslogging engine. I've found installing syslog-ng on red hat can be tricky.

A: That is exactly why we provide a container as the packaging. This is provides a very easy-to-deploy runtime that does not depend on the syslog engine in the distro (which is often very old).


Q: Have you tried sending SEP to this?

A: Yes, we have OOTB support for SEP syslog and a new TA.


Q: Can you filter by msg header?

A: Yes -- we have a number of sources that we support "Out of the Box" where we filter on the header as well as message contents.


Q: I have a case were I have multiple logs being sent to syslog and are being written to one file = sourcetype hijacking.

A: SC4S is perfect for fixing this.


Q: Can you filter and black list by device?

A: Yes -- via "compliance" overrides


Q: Does SC4S allow for forwarding of syslog events to other endpoints?

A:  Yes, alternate HEC endpoints can be configured directly, and alternate syslog-ng destinations can be created and used.


Q: Is this a free solution or charged based on volume?

A: SC4S is free to use for all Splunk customers.


Q: Can that forwarding be selective, i.e. only a subset of particular sources?

A: Yes


Q: Can we slap SC4S on existing syslog servers so we don't have to redirect all the syslog sources?

A: You may be able to deploy SC4S using your existing HW however it would replace the need for your existing syslog services for the sources in question.


Q: Are there defined limits as to how many messages/sec SC4S can handle?

A: The documentation includes information on sizing; very "back of the napkin" scale shows a 16/32 box can handle ~6 TB/day -- a _lot_. But keep in mind that syslog is meant for edge collection, so avoid the urge to centralize.


Q: Can this be used in a multi cluster env?

A: Yes


Q: What happens to the priority indicator at the beginning of the syslog event?
A: These are included as indexed fields in the event for your use.


Q: We have a requirement to store syslog for varying amounts of time. Are there varying amounts of storage available, or would we have to export to a SAN?

A: If you have local storage needs, data can be stored on a mounted directory (of your choice) to the container.


Q: "Bring your own" supported on RHEL 7, or 8 only?

A: The supported OS and runtime combinations are here. https://splunk-connect-for-syslog.readthedocs.io/en/master/gettingstarted/


Q: When BYO, is the data "cleanup" shown initially also part of the components available?

A: Yes -- it contains the identical syslog-ng configs that are included in the container version -- all the "secret sauce" is there.


Q: Does this solution currently work on splunk cloud?

A: The sc4s collector itself is deployed on prem or in your own cloud. But it is tailor-made for _use_ with Splunk Cloud, which supports HEC out of the box.




Splunk Employee
Splunk Employee

We also want to make sure you have these additional resources for your journey:

  1. Splunk Connect for Syslog Documentation
  2. Take Control of Port 514!: Taming the Syslog Beast .conf19 Paper
  3. Splunk Connect for Syslog: Turnkey and Scalable Syslog GDI – Part 1 Blog
  4. Splunk User Group Slack Channel #splunk-connect-for-syslog
  5. Splunk Administrator Training Courses