Splunk Dev

How to create a logger to send Syslog data into Splunk

POR160893
Builder

Hi,

I am currently working on developing a logger  in Python that would send Syslog data into Splunk.

So, I want to create a logging interface or an abstract class that would create a logging class for the Syslog data, e.g. log -> write_log.

The log->write_log is then overrides the abstract method with the concrete method for syslog->write_log.

How can I start this task? What Python library would I need? Like splunklib or splunk_handler?


Thanks!

Labels (2)
0 Karma

POR160893
Builder

Hey,

At the moment, I have been trying to connect to Splunk but appears I am getting a connection issue:

POR160893_0-1637282284358.png

 

The error message is as follows:

POR160893_1-1637282331518.png

Any advice on how to approach this please?



0 Karma

sloshburch
Splunk Employee
Splunk Employee

I think everyone here is encouraging you to use existing supported methods to do this rather than write your own. The outcome is that if there is a problem you will have to support your custom solution and Splunk Support will not be able to help you. If you use one of the existing approaches then you have the help of Splunk Support and eng there as well as a large community of users experienced with the already-existing approaches.

Nonetheless, if you are trying to send network communication then there must be a listener open on that port. In Splunk those are called TCP/UDP/HEC Inputs. https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/Monitornetworkports

0 Karma

PickleRick
SplunkTrust
SplunkTrust

So you decided to write a syslog-based logger.

Ok, should more or less work but it's not splunk-specific and you have to prepare for it properly on splunk's side.

I assume that you didn't. That's why you're getting your errors. Do you have any input (either sc4s or a direct splunk input on port 514)?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

I'm  a bit confused - are you trying to log from your own application or want to build application to receive syslog and write to splunk?

If the former - why you mention syslog? If the latter - there are already working solutions for that.

0 Karma

sloshburch
Splunk Employee
Splunk Employee

I agree with these questions for more information. I think a good logical flow to assume is that the private software produces syslog and sends it to Splunk. Splunk will then receive it using something like https://splunk.github.io/splunk-connect-for-syslog. The point I'm making is that you don't want to be natively trying to write into Splunk to create the log events - rather, use the very mature interfaces that are set up for this. Additionally, if you are creating the log events yourself then I would recommend some of the less lossy approaches like log files or HTTP events. https://dev.splunk.com/enterprise/docs/devtools/httpeventcollector

0 Karma

POR160893
Builder

So, the logs would be coming from a Python Django REST API. We want to log what was received from this Django API onto Splunk, I just need now to ingest the Syslog into Splunk using Python but I don't know how.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Again - syslog is a separate layer.

From application perspective you should just use a common logging framework (if I remember correctly, Python has one built-in). Then you - depending on the target logging method - should just create proper handler either emiting syslog messages (then you can use rsyslog/SC4S or even splunk's built-in tcp/udp listener) or a handler sending events to HEC endpoint. Or one writing to files (then you can set your UF to read from files).

 

0 Karma

sloshburch
Splunk Employee
Splunk Employee

I'm not familiar with using that but if the logs are already available through a REST API then creating a modular input would still be a solid option. Again, this is a bit out of my domain but it sounds like if you can push the data to Splunk, then HEC would be best. If you must pull the data into Splunk then a Modular Input (easily built with the Splunk Add-on Builder). If you can write to a log file then a Splunk Universal Forwarder can help. Finally, syslog with Splunk Connect 4 Syslog, assuming you can push the data out and communicate over standard syslog. All of the proper names I mentioned area easily found online with a web search.

I hope that helps!

Also, if it's relevant: https://community.splunk.com/t5/Developing-for-Splunk-Enterprise/Writing-own-REST-API-in-Splunk-app/...

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...