topic Filtering out header-only "events" after using MultiKV in Getting Data In

Filtering out header-only "events" after using MultiKV

FunPolice — Fri, 26 Nov 2010 11:15:49 GMT

I've been tweaking the *NIX app by adding some charts with queries such as:

index="os" sourcetype="who" host=$host$ | multikv | timechart count by USERNAME

The resulting chart has a large number of events where the USERNAME is NULL. This appears to be because the MultiKV command only filters out the header when there is a body in the event. If the "who" event only contains a header the MultiKV leaves it in the result set as an event.

What is the best way to get rid of these unwanted header "events"? I would prefer to index them, as then there is a contiguous record from the who script. I could use

index="os" sourcetype="who" | multikv | search _raw!="*USERNAME        LINE        HOSTNAME                                  TIME*"

however there are likely many other similar situations so a more generic solution would be helpful.

Is there a better way?

Thanks.

Re: Filtering out header-only "events" after using MultiKV

Ledion_Bitincka — Sat, 27 Nov 2010 11:41:57 GMT

a slightly better approach would be:

index="os" sourcetype="who" host=$host$ | multikv | search USERNAME=* | timechart count by USERNAME

basically filter out all results that are missing the USERNAME field

Re: Filtering out header-only "events" after using MultiKV

gkanapathy — Tue, 30 Nov 2010 06:06:12 GMT

I just don't get this behavior. multikv for me simply doesn't return the header rows, so I don't get this problem.

Re: Filtering out header-only "events" after using MultiKV

FunPolice — Mon, 23 May 2011 22:45:18 GMT

I've upgraded to 4.2 and will try again with the universal forwarder as soon as I get a chance. Stay tuned!