All Apps and Add-ons

isn't there a native way to ship journald logs to Splunk?

eddiemashayev
Path Finder

Hey All,

I saw this article: https://www.splunk.com/blog/2015/04/30/integrating-splunk-with-docker-coreos-and-journald.html
It look like an overkill for a simple task, isn't there a native way to ship journald logs to Splunk?

Please assist.

Labels (1)
Tags (2)

tgurantz_splunk
Splunk Employee
Splunk Employee

I know this is an older post, but I didn't see anyone yet mentioning that this looks to have been solved by journald support in recent versions of Splunk (starting in Splunk 8.1), correct?

https://docs.splunk.com/Documentation/Splunk/latest/Data/CollecteventsfromJournalD

2cB2sVJCSp
Observer

I would say this partly covers shipping systemd journal logs to splunk.

What I would really love is for splunk to be able to accept data sent by systemd-journal-upload ( https://www.freedesktop.org/software/systemd/man/latest/systemd-journal-upload.service.html ).

That way you'd not need a forwarder on any popular systemd distribution anymore. You could just use systemd.

0 Karma

alastor
Path Finder

I've posted about this in another question. this seems to have come up multiple times in questions and people are not upvoting or saying me too on these journald issues.

Splunk provides the TA for Unix / Linux but it's so out of date and doesn't collect these critical logs for systems using systemctl.

Expecting every customer to come up with some silly way of dumping journald to a file and importing it is absurd. We all need to start yelling at our Splunk reps about this issue.

0 Karma

hettervik
Builder

Hi. I'm looking into gathering journal logs as well. I think the solution presented in the link you provide will not work. When the script for trimming the written to journal file start kicking in, Splunk will no longer be able to recognize the file, and will start to read it all over again from the start, if I'm not mistaken.

Another solution to the problem is presented here: https://answers.splunk.com/answers/554744/scripted-input-last-event-lost-journald.html

However I'm not sure if constantly tailing the journal is a good solution performance wise. Also, with this solution, if the UF goes down due to updates or failure, you will lose logs.

I think the best way to go is to somehow use the cursor field to keep track of where you are in the journal, and start reading from last seen cursor when the UF starts, but I haven't yet found a good script for this.

Have you found any better solutions?

0 Karma

logloganathan
Motivator
0 Karma

eddiemashayev
Path Finder

Can you elaborate more? I don't see any reference for journald or anything similar.

0 Karma

logloganathan
Motivator

you can find this in inputs.conf
[monitor:///var/log/journal]

0 Karma

alastor
Path Finder

Can someone from Splunk PLEASE give a serious answer to this question?

Journal logs from systemctl are not something that it makes sense to ignore, and as I've said in another post about this, should be part of the default TA_NIX support at this point, most distros use this and have for YEARS now.

Asking users to come up with some whack unsupported script to dump logs into a file is BS.   you have plenty of scripted inputs in the add on for nix already, solve this in a standard way!

If you want to opensource the development of the nix TA, put it on github or something and start letting customers do it.  Having it fragmented and undocumented is dumb.

a02719
Engager

totally agree, this is an absolut hoax, there i can use also json-file logdriver if i want to duplicate or triplicate the space i need for logs on every host ..

0 Karma

eddiemashayev
Path Finder

Hey,
Thanks for the reply, but it's not a native way, as I need to take journald logs and redirect them to file for example /var/log/journal

0 Karma
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...