Splunk Search

Is there a bug in dedup?

cmeo
Contributor

I have the following query which almost does what I want:

sourcetype="cisco_wsa_squid"
| lookup teamlookup cs_username
| search tl_logon != ""
| stats count(s_hostname) AS Hits, sum(sc_bytes) AS Bytes by tl_display, m_display, s_hostname
| eval MBytes=round((Bytes/(1024*1024)),2)
| fields - Bytes
| dedup tl_display,m_display sortby tl_display, m_display, -num(MBytes) keepevents=t

What this is supposed to do is summarise hits and traffic by site, team member and team leader. I obviously don't want see line items except for the combination of site, Hits and MBytes. teamlookup just works out what team the user belongs to. Not all staff are in teams for splunk reporting purposes.

But I want to limit my output to 15 rows per group. Both dedup 15 ... and dedup ... limit=15 produce a broken result which is correct for the first 15 rows, then omits the leading field but shows all the events that I'm trying to suppress.

Has anyone else seen anything like this? Is is a bug?

Splunk v 4.1.4 on Linux.

Tags (1)
0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

This is not a bug. The "keepevents" flag to dedup causes it to not discard the later events, but rather discard just the duplicate fields. If you want to discard the rows, omit "keepevents=t".

Stephen_Sorkin
Splunk Employee
Splunk Employee

It should just be "... | dedup 15 tl_display m_display sortby -num(MBytes)". This will keep the 15 first (as sorted by descending MBytes) event for each combination of tl_display and m_display. Only events will all the fields will be returned.

cmeo
Contributor

I tried without keepevents=t and with both dedup 15... and dedup ... limit=15

tl_display and m_display are no longer deduped, but I do only see 15 lines. I still think it is broken. Doesn't seem to like an integer argument much.

0 Karma

cmeo
Contributor

Is there any way to post screenshots here? What I'm seeing is definitely not right, but I need to show it.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Telemetry Pipeline Management

Balancing Scale and Spend: Gaining Control Over High-Volume Metrics in Splunk Observability Cloud As ...

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...