Splunk Search

How Report Any Host That Hasn't Had an Event From Source="/var*" in "X" Minutes

SplunkLunk
Path Finder

Greetings,

I want to report on any Linux system that hasn't had an event in /var* for 30 minutes. I was going to use Source="/var/log/messages" but our admins told me that they want anything below /var to be reported on. I tried using the metadata command but that didn't get me anywhere. Does anyone have any suggestions? Thanks.

0 Karma

to4kawa
Ultra Champion
| tstats count as Counts where source="/var/*" by source host
| inputlookup append=t your_host.csv
| stats values(Counts) as count by source host
| fillnull count
| where count=0

your_host.csv:

host,source
A,/var/log/messages
A,/var/log/
B,/var/log/messages
B,....

Field name is case sensitive. Source is wrong.
I don't know what's gold. so both source and host is aggregated.

0 Karma

SplunkLunk
Path Finder

Thanks. Will a wildcard in the CSV file work? Could the source be /var/log/*

0 Karma

to4kawa
Ultra Champion

between no2 and no3,

| rex mode=sed field=source "s/\/\w+/\/*/3"

to force them to match.

0 Karma

SplunkLunk
Path Finder

Thanks but I must be doing something wrong. I've added a lookup file with the format:

host, source
host1, /var/log*
host2, /var/log*

I've done your search:

| tstats count as Counts where index=[my index] source="/var/log*" by host
| inputlookup append=t May_Linux_Hosts.csv
| rex mode=sed field=source "s/\/\w+/\/*/3"
| stats values(Counts) as count by source host
| fillnull count
| where count=0

And the output I get is:

source, host, count
/var/log/, host1,0
/var/log/
, host2,0
.
.
.

So it's like it's not treating the * as a wildcard. Any idea what I'm doing wrong?

0 Karma

to4kawa
Ultra Champion

your comment: /var/log/* now: var/log*

| rex mode=sed field=source "s/\/\w+/\/*/3"
| rex mode=sed field=source "s/(\/var\/log).*/\1*/"

We use regular expressions, so even the slightest difference won't work.

0 Karma

SplunkLunk
Path Finder

Thanks for the help. Still producing same output with the following search. I guess I don't know enough about regex to troubleshoot. I was even going to regex101.com to test without any luck:

| tstats count as Counts where index=ahc_os source="var/log*" by host
| inputlookup append=t May_Linux_Hosts.csv
| rex mode=sed field=source "s/(\/var\/log)./\1/"
| stats values(Counts) as count by source host
| fillnull count
| where count=0

0 Karma

SplunkLunk
Path Finder

It jacked the formatting of the rex line up. I cut and paste exactly like you showed, but it doesn't look that way in my response.

0 Karma

to4kawa
Ultra Champion

check your source value after rex

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...