Splunk Search

Stats count by when field exists, otherwise use another

DEAD_BEEF
Builder

I am trying to create a dashboard that graphs the parsing queue size for a HF by ingest_pipe. I noticed that most of these logs have that field but some don't (i'm not sure why).

sample logs

06-03-2020 12:21:30.964 -0400 INFO  Metrics - group=queue, name=parsingqueue, max_size_kb=512, current_size_kb=0, current_size=0, largest_size=2, smallest_size=0

06-03-2020 12:21:27.144 -0400 INFO  Metrics - group=queue, ingest_pipe=3, name=parsingqueue, max_size_kb=6144, current_size_kb=0, current_size=0, largest_size=2, smallest_size=0

06-03-2020 12:21:27.142 -0400 INFO  Metrics - group=queue, ingest_pipe=2, name=parsingqueue, max_size_kb=6144, current_size_kb=0, current_size=0, largest_size=11778, smallest_size=0

current SPL

index=_internal host=$hostToken$ group=queue name=parsingqueue
| timechart avg(current_size_kb) by ingest_pipe

I can't modify the search with ingest_pipe=* because I have tokenized the host field in the search and some of my HF's only have 1 ingest pipe. In that scenario, there is no ingest_pipe field at all so hardcoding that into the search will result in 0 results when the HF only has 1 pipeline.

The solution I came up with is to count the # of events where ingest_pipe exists (yesPipe), count the # of events where it does not exist (noPipe), and assign my count by foo value to the field that is greater. If yesPipe is greater, count by ingest_pipe, else count by host. I don't have the query for these counts and checks.

Alternatively, I thought I could use a lookup table that has a "count by field" column, where per host I simply specify either ingest_pipe or host to count by.

I feel like there is an easy solution and I'm overthinking it. Any ideas?

Labels (3)
Tags (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Try fillnull.

index=_internal host=$hostToken$ group=queue name=parsingqueue
| fillnull value=0 ingest_pipe
| timechart avg(current_size_kb) by ingest_pipe
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Try fillnull.

index=_internal host=$hostToken$ group=queue name=parsingqueue
| fillnull value=0 ingest_pipe
| timechart avg(current_size_kb) by ingest_pipe
---
If this reply helps you, Karma would be appreciated.

DEAD_BEEF
Builder

So when ingest_pipe is not there, it gives it a value of 0, and when it is present, it does nothing?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That is what fillnull does, yes.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...