topic Re: Want to ignore some fields in Splunk Search

Want to ignore some fields

Ashish0405 — Mon, 16 Dec 2024 22:43:37 GMT

Hi Team,

In below query I don't want to show up the result as "Up" in state_to field, I just want to see data with down state , also it not possible to use (exclude operator state_to!=Up ) because it is showing all down results which is not my aim.

Please help and suggest !

My query :

index="network" %BGP-5 *clip* | dedup src_ip | stats count by state_to,Device_name,src_ip

with exclude result: ( which I dont want)

My expected result/AIM : it will just show the result the devices which are down at the moment and dont want see the UP result

AIM : With the help of query

Re: Want to ignore some fields

richgalloway — Mon, 16 Dec 2024 19:12:45 GMT

Try excluding Up states at the end rather than at the beginning.

index="network" %BGP-5 *clip* | dedup src_ip | stats count by state_to,Device_name,src_ip | where state_to!="Up"

Re: Want to ignore some fields

PickleRick — Mon, 16 Dec 2024 21:46:35 GMT

Overall logic of your search is flawed. You firstly remove a lot of data with dedup and then try to stats over hugely incomplete data set.

What is it you're trying to do (in your own words, without SPL)?

Re: Want to ignore some fields

Ashish0405 — Tue, 17 Dec 2024 06:53:01 GMT

Wow, the expected result popped up !!!

Thanks !!!!, I will do some testing

Re: Want to ignore some fields

PickleRick — Tue, 17 Dec 2024 07:03:54 GMT

Again - it might be expected but is it the correct result?

Consider this example run-anywhere search

| makeresults format=csv data="a,b,c
2,3,3
1,2,3
2,2,2
1,3,2"
| dedup c
| stats count by a b c

Run it, write down the results.

Now run the same search but with a reordered input mockup data

| makeresults format=csv data="a,b,c
1,2,3
2,3,3
1,3,2
2,2,2"
| dedup c
| stats count by a b c

As you can see, the data on which you're operating is the same, just in a different order and the results are completely different. So you might want to rethink your search logic.

Re: Want to ignore some fields

Ashish0405 — Tue, 17 Dec 2024 07:36:14 GMT

Would you be able to help me on one more thing please !!!

I have a Number display dashboard which represent the BGP flap details as # Device_name & #BGP peer IP , however I cannot add the timing when the BGP flap on Number display

Current Query : index="network" %BGP-5 *clip* | rex field=_raw "^(?:[^ \n]* ){4}(?P<Device_name>[^:]+)" | dedup Device_name,src_ip | stats count by Device_name,src_ip,state_to | eval primarycolor=case(state_to="Down", "#D93F3C", state_to="Up", "#31A35F") | eval secondarycolor=primarycolor

Is there something we can add to display flap time in the same number display

Re: Want to ignore some fields

Ashish0405 — Tue, 17 Dec 2024 07:49:53 GMT

Yes, thank you for these details,

I guess if I sort the output with time ( # sort _time) the result will be rearranged as per date & time, AM I CORRECT ?

So if with the help of sort _time data get re-arranged then the latest one result will be either #UP or #DOWN, then the AIM is achieved

Re: Want to ignore some fields

PickleRick — Tue, 17 Dec 2024 07:51:52 GMT

This is a different issue - more visualization-related. Please post it as a new thread in a proper forum section to keep this forum tidy 🙂

Re: Want to ignore some fields

Ashish0405 — Tue, 17 Dec 2024 08:02:22 GMT

Thanks I will post in new thread

Re: Want to ignore some fields

PickleRick — Tue, 17 Dec 2024 11:29:27 GMT

Well, "correct" depends on what you want to achieve. Often dedup is not needed if you're going to stats the data right after it. And it's a tricky command and often misunderstood and misused. It filters out from your results all further events with a particular value of a given field (or a set of values if you use it on more fields) regardless of what the remaining content of those events are.

So if you had, for example, logs containing fields criticality (being one of INFO, WARN, DEBUG, ERR or CRIT) and message after using | dedup criticality you'd only get one INFO, one DEBUG and so on - the first one Splunk encountered in your data. You'd lose all subsequent INFOs, DEBUGs and so on even though they had different message value. So you'd be aware that - for example - there was a CPU usage spike but wouldn't know that your system was also out of disk space and over the temperature threshold.

Dedup is really rarely useful. For me it works only as an "extended head".

Re: Want to ignore some fields

Ashish0405 — Wed, 18 Dec 2024 05:57:11 GMT

Thank you for the details, this will help me with current dashboard