Splunk Search

How Report Any Host That Hasn't Had an Event From Source="/var*" in "X" Minutes

SplunkLunk
Path Finder

Greetings,

I want to report on any Linux system that hasn't had an event in /var* for 30 minutes. I was going to use Source="/var/log/messages" but our admins told me that they want anything below /var to be reported on. I tried using the metadata command but that didn't get me anywhere. Does anyone have any suggestions? Thanks.

0 Karma

to4kawa
Ultra Champion
| tstats count as Counts where source="/var/*" by source host
| inputlookup append=t your_host.csv
| stats values(Counts) as count by source host
| fillnull count
| where count=0

your_host.csv:

host,source
A,/var/log/messages
A,/var/log/
B,/var/log/messages
B,....

Field name is case sensitive. Source is wrong.
I don't know what's gold. so both source and host is aggregated.

0 Karma

SplunkLunk
Path Finder

Thanks. Will a wildcard in the CSV file work? Could the source be /var/log/*

0 Karma

to4kawa
Ultra Champion

between no2 and no3,

| rex mode=sed field=source "s/\/\w+/\/*/3"

to force them to match.

0 Karma

SplunkLunk
Path Finder

Thanks but I must be doing something wrong. I've added a lookup file with the format:

host, source
host1, /var/log*
host2, /var/log*

I've done your search:

| tstats count as Counts where index=[my index] source="/var/log*" by host
| inputlookup append=t May_Linux_Hosts.csv
| rex mode=sed field=source "s/\/\w+/\/*/3"
| stats values(Counts) as count by source host
| fillnull count
| where count=0

And the output I get is:

source, host, count
/var/log/, host1,0
/var/log/
, host2,0
.
.
.

So it's like it's not treating the * as a wildcard. Any idea what I'm doing wrong?

0 Karma

to4kawa
Ultra Champion

your comment: /var/log/* now: var/log*

| rex mode=sed field=source "s/\/\w+/\/*/3"
| rex mode=sed field=source "s/(\/var\/log).*/\1*/"

We use regular expressions, so even the slightest difference won't work.

0 Karma

SplunkLunk
Path Finder

Thanks for the help. Still producing same output with the following search. I guess I don't know enough about regex to troubleshoot. I was even going to regex101.com to test without any luck:

| tstats count as Counts where index=ahc_os source="var/log*" by host
| inputlookup append=t May_Linux_Hosts.csv
| rex mode=sed field=source "s/(\/var\/log)./\1/"
| stats values(Counts) as count by source host
| fillnull count
| where count=0

0 Karma

SplunkLunk
Path Finder

It jacked the formatting of the rex line up. I cut and paste exactly like you showed, but it doesn't look that way in my response.

0 Karma

to4kawa
Ultra Champion

check your source value after rex

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...