I am trying to create a dashboard that graphs the parsing queue size for a HF by ingest_pipe
. I noticed that most of these logs have that field but some don't (i'm not sure why).
sample logs
06-03-2020 12:21:30.964 -0400 INFO Metrics - group=queue, name=parsingqueue, max_size_kb=512, current_size_kb=0, current_size=0, largest_size=2, smallest_size=0
06-03-2020 12:21:27.144 -0400 INFO Metrics - group=queue, ingest_pipe=3, name=parsingqueue, max_size_kb=6144, current_size_kb=0, current_size=0, largest_size=2, smallest_size=0
06-03-2020 12:21:27.142 -0400 INFO Metrics - group=queue, ingest_pipe=2, name=parsingqueue, max_size_kb=6144, current_size_kb=0, current_size=0, largest_size=11778, smallest_size=0
current SPL
index=_internal host=$hostToken$ group=queue name=parsingqueue
| timechart avg(current_size_kb) by ingest_pipe
I can't modify the search with ingest_pipe=*
because I have tokenized the host field in the search and some of my HF's only have 1 ingest pipe. In that scenario, there is no ingest_pipe
field at all so hardcoding that into the search will result in 0 results when the HF only has 1 pipeline.
The solution I came up with is to count the # of events where ingest_pipe exists (yesPipe), count the # of events where it does not exist (noPipe), and assign my count by foo
value to the field that is greater. If yesPipe is greater, count by ingest_pipe
, else count by host
. I don't have the query for these counts and checks.
Alternatively, I thought I could use a lookup table that has a "count by field" column, where per host I simply specify either ingest_pipe
or host
to count by.
I feel like there is an easy solution and I'm overthinking it. Any ideas?
Try fillnull
.
index=_internal host=$hostToken$ group=queue name=parsingqueue
| fillnull value=0 ingest_pipe
| timechart avg(current_size_kb) by ingest_pipe
Try fillnull
.
index=_internal host=$hostToken$ group=queue name=parsingqueue
| fillnull value=0 ingest_pipe
| timechart avg(current_size_kb) by ingest_pipe
So when ingest_pipe is not there, it gives it a value of 0, and when it is present, it does nothing?
That is what fillnull
does, yes.