topic Re: Need help in append in Splunk Search

Need help in append

srinivasgowda — Fri, 05 Mar 2021 08:02:06 GMT

Hello all,

I am facing an issue in appending an query. Here my objective is to update the kv store with the list of servers, alert_flag(if the alert has been raised) and count(number of times the server has created an event). Below is the query that I have used.

When the above is ran everytime the same host is updated and also added in the new row, however, I need a single update of the count and alert_flag for a host. The data is pushed to the kv store as below by a new increase in the count.

_time alert_flag count source_host
2021-03-05 13:01:50 0 1 Server 1
2021-03-05 13:01:50 0 1 Server 2
2021-03-05 13:01:50 0 1 Server 3
2021-03-05 13:01:53 0 2 Server 1
2021-03-05 13:01:53 0 2 Server 2
2021-03-05 13:01:53 0 2 Server 3

However, I am looking for the data to be updated in the KV store like below.

_time alert_flag count source_host
2021-03-05 13:01:53 0 2 Server 1
2021-03-05 13:01:53 0 2 Server 2
2021-03-05 13:01:53 0 2 Server 3

Please guide me through this.

Regards

Re: Need help in append

manjunathmeti — Fri, 05 Mar 2021 08:28:26 GMT

hi @srinivasgowda,

Use stats with latest function to get latest values by source_host.

index= index | lookup source_host_kvstore_001 source_host OUTPUT source_host as temp_source_host count alert_flag | eval count=if(isnull(count),1,count+1) | eval alert_flag = if(isnull(alert_flag),0,if((alert_flag=1),1,0)) | eval _time=now() | fields _time source_host alert_flag count | stats latest(_time) as _time latest(*) as * by source_host | outputlookup source_host_kvstore_001 append=true

Re: Need help in append

srinivasgowda — Fri, 05 Mar 2021 08:35:28 GMT

Hello @manjunathmeti ,

Thanks for the quick response. This is still giving the same result by adding new rows for the same source_host in the kvstore. I am looking to have a singe row for each source_host and just the count to increase everytime there is an event from the source_host.

Regards

Re: Need help in append

ITWhisperer — Fri, 05 Mar 2021 08:36:21 GMT

You need to use append=false

Re: Need help in append

srinivasgowda — Fri, 05 Mar 2021 11:05:59 GMT

This will update the count for the source_host, however, if a new source_host come in then the existing data in the kvstore would be deleted.

Re: Need help in append

ITWhisperer — Fri, 05 Mar 2021 11:16:24 GMT

Can you append / union the current contents of the store so your search includes everything you want before you output it?

Re: Need help in append

srinivasgowda — Fri, 05 Mar 2021 11:19:42 GMT

Yes, append=false works as long as the same set of source_host is repeated in every run, but if in a run there is events from just 1 source_host then the remaining in the kvstore would be deleted updating just the one that was currently present.

Re: Need help in append

ITWhisperer — Fri, 05 Mar 2021 11:35:21 GMT

Exactly, so use inputlookup as part of the search to append or union the current contents of the keystore

Re: Need help in append

srinivasgowda — Fri, 05 Mar 2021 11:40:41 GMT

inputlookup does not work after using outputlookup.

Re: Need help in append

ITWhisperer — Fri, 05 Mar 2021 12:32:31 GMT

Use inputlookup (to get current contents) before outputlookup (to write full set)