topic How to squash host and source fields? in Getting Data In

How to squash host and source fields?

Omar — Tue, 16 Aug 2022 13:08:45 GMT

Dear Splunkers,

I am having an issue with the process of squashing fields. When searching for events with no hosts or source I don't get any results:

index=<my_index> 
| where isnull(source)

Does Splunk drop events after being squashed? Because logically, there should be events on my index that are missing the field host and source.

Re: Squashing host and source fields

gcusello — Tue, 16 Aug 2022 06:21:37 GMT

Hi @Omar,

at first is very strange that the source field is null because every event must have a value in this field.

Anyway, if you want to search events without values in the source field, please try this:

index=<my_index> NOT source=*

Ciao.

Giuseppe

Re: Squashing host and source fields

Omar — Tue, 16 Aug 2022 06:43:25 GMT

Hello @gcusello,

Thank you for your response.

Actually, yes, this could happen due to a process of squashing

when a certain threshold is reached indexers drop (host, source) fields to avoid explosion in memory/processing overhead.

what confuses me is I am unable to find those events, so I'm wondering if Splunk is dropping the entire events or just those fields.

Bellow search shows if you have this issue or not. This only works with large indexes:

index=_internal source=*license_usage.log* type="Usage" idx="my_index"
| eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h)
| eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s)
| eval st=if(len(st)=0 OR isnull(st),"(UNKNOWN)",st)
| fields _time,b,h,st
| bin _time span=1d
| stats sum(b) AS volume by h, _time,st
| stats avg(volume) AS avgVolume max(volume) AS maxVolume by h,st
| eval avgVolumeGB=round(avgVolume/1024/1024/1024,3)
| eval maxVolumeGB=round(maxVolume/1024/1024/1024,3)
| fields h,st, avgVolumeGB, maxVolumeGB
| rename avgVolumeGB AS "average" maxVolumeGB AS "peak",st AS "sourcetype", h AS "hostname"
| sort - average
| head 10

Re: Squashing host and source fields

gcusello — Tue, 16 Aug 2022 06:49:24 GMT

Hi @Omar,

it's the first time I see this behavior, I found that when there's a congestion (for full queues) there's a delay in internal logs indexing but I never found that host an source fields are dropped!

Anyway, I hint to analyze why there's this congestion and found a solution, maybe it's a too slow storage or maybe you need more resources for your servers or there's a queue problem for a wrong configuration.

Anyway, open a ticket to Splunk Support for this.

Ciao.

Giuseppe