topic Re: How to forward logs from syslog server to splunk so that it is recognizable via their sourcetype? in Deployment Architecture

How to forward logs from syslog server to splunk so that it is recognizable via their sourcetype?

tan_junyuan — Wed, 30 Sep 2020 02:33:18 GMT

eg. I have many logs forwarded to the syslog server.

I intend to install a universal forwarder on that syslog server to forward to splunk.

However once forwarded to Splunk, what will be the sourcetype?

Can my checkpoint server logs be recognized as sourcetype=cp_log in Splunk or is it syslog?
I tried just uploading the log file in splunk with sourcetype=cp_log, it does not recognize the format.

Log entry format is as follows:
Checkpoint xxxx -[action:" xx",flags:"xxx", ifdir:"xx",etc ]

Re: How to forward logs from syslog server to splunk so that it is recognizable via their sourcetype?

ololdach — Wed, 16 Oct 2019 08:57:23 GMT

Hi,
this is a very general question and you are looking at a steep learning curve, be prepared to do some reading, testing and learning yourself and please don't get frustrated! You will find that after a few "ah-ha"-moments it will get a lot easier and suddenly make sense.

To get you started on your journey:
1. Configure your syslog server so that every host will get his own subdirectory: /var/log/<hostname>/...
2. A sourcetype in Splunk is just a name to a set of parameters in the configurations that help Splunk to tie metadata together. Splunk uses this sourcetype mainly to: Determine event breaking, timestamp and field extraction and transformations of the raw data both at index and at query time. Some well known sourcetypes are "built-in" whereas others need to be defined by you or are predefined inside an app. For Checkpoint logs, please check https://splunkbase.splunk.com/apps/#/search/checkpoint/ There are several apps that define sourcetypes based on CP logs
3. After you have a sourcetype defined, you need to create an inputs.conf stanza to monitor the log files and tie this file to a certain sourcetype that is being defined in the app that you installed or that you have defined yourself. There are ways to ingest all logs with a "dummy" sourcetype and do the actual matching afterwards. However, that method has some tremendous drawbacks and should be considered advanced and its use limited to very special use cases.
4. Lastly, you need to get that inputs.conf file onto your forwarder by copying it into the right directory or by using a deployment server. There is some great documentation on this topic at: https://docs.splunk.com/Documentation/Forwarder/7.3.2/Forwarder/Abouttheuniversalforwarder

If you get stuck, please do post a specific question with the error or the problem that you encounter and I am certain, we'll be able to help. - Hope it helps Oliver

Re: How to forward logs from syslog server to splunk so that it is recognizable via their sourcetype?

tan_junyuan — Mon, 21 Oct 2019 10:16:10 GMT

if it is a normal syslog server, that forward whatever raw logs it receive to splunk, shouldn't have an issue.

However, if it is a Kiwi syslog server that forward to splunk, there are many headers added that cause Splunk not to recognize the log format.

YYYY-MM-DD HH:mm:ss Local0.Info xx.xx.xx.xx 1 YYYY-MM-DDThh:mm:ssZ CheckPoint xxxx - []

Any ideas how to solve this issue?