Dashboard with /var/log/sudo.log, /var/log/secure and /var/log/audit/audit.log Events

[genesiusj]

Hello,
I am working on dashboard for our Linux admins. They require being able to view all events from /var/log/sudo.log, /var/log/secure and /var/log/audit/audit.log for a single user on a single host.

I am not well-versed in Linux and I am having difficulty creating the proper rex commands for my SPL.

For example, here is an event from the /var/log/secure.

Aug  1 13:33:59 server10 groupadd[51032]: group added to /etc/gshadow: name=splunk

How I can find out which user added this group? I believe I could group by the PID. But then I would have to track every PID that is spawned off of the original and subsequent PIDs.

Before I delve down this very complex path, has anyone worked on this idea before? Would you mind sharing your SPL for this?

Thanks in advance and God bless,
Genesius

[jpolvino]

Here is a post with some potentially helpful information: Splunk App for Unix and Linux: How to extract user field from /var/log/secure?

[woodcock]

You do not need to do all of this yourself. Try this app:
https://splunkbase.splunk.com/app/3476/

[genesiusj]

@woodcock
I am not sure if we will be able to use these, except in our test instance. We have a policy; no software can be installed in production unless it is supported by the vendor. Even though this is Splunk AppInspect Passed, it is not supported by Splunk, or a known third party, we won't install in production.
Thanks for your help.
Enjoy your weekend.
God bless,
Genesius

[woodcock]

First of all, most Splunk apps, including this one, are not software; they are bundles of configuration files. Second, a sensible approach for apps such as this one (especially with open-source licenses) would be to download the app, open it up, and rip out the configurations that you need and deploy them outside of the app. We are only talking about RegEx definitions in this case.

[genesiusj]

@woodcock
Thanks. I know they are bundles of configuration files. To senior management, they are software not developed by us, supported by us, or supported by any of our approved vendors.
Having said that, we will look at installing these on our test instance and extract the necessary rex/regex statements. Our test instance is also used for developing SPL and XML for the user community. Replacing the Splunk_TA_nix app may cause some issues with them.
I was hoping there was an available source (such as Go.Splunk) that would have this code and that I would be able to tweak for our purposes.

I have started down the SPL path myself and to append all three of these log files into one table is a bear, because there is no one field that exists in all events. Not only within all three logs, but within each log itself. Small wonder why Splunk hasn't been able to develop a useful Linux app, like they did with the Splunk App for Windows Infrastructure.

Thanks again for your help. I will create a new post with my findings.

Enjoy your weekend and God bless,
Genesius

[vliggio]

Go.Splunk is just a collection of community provided Splunk queries. Why is that more supported than something from apps.splunk.com? The suggested app is written by Doug Brown who works for RedHat - if anyone can write an app to interpret Linux logs, I'd expect it to be a RedHat employee!

[genesiusj]

If it is written by a RedHat employee than RedHat should look into supporting it.
Getting into the "whys" of my organization's policies isn't really the direction I want to go. If someone has some information to assist with the issue I posted, great.
I don't write policy, I follow policy.

Update: I have started down the path to joining these logs together, and it has been an education. I will share more once I am done.

Thanks and God bless,
Genesius

[youngsuh]

Could you update us on the dashboard?