Splunk Search

Why isnt the dedup command not working in some fields?

payton_tayvion
Path Finder

Im having an issue where my contact field and l2 field is showing duplicates of the same name and when i use the dedup command nothing happens. Here's the command:
| tstats count(Vulnerabilities.signature) as "Total Vulnerabilities" from datamodel="Vulnerabilities" where (nodename="Vulnerabilities" Vulnerabilities.severity="high" OR Vulnerabilities.severity="critical" Vulnerabilities.dest_bunit IN (*) Vulnerabilities.property IN (*)) by Vulnerabilities.signature Vulnerabilities.property
| rename Vulnerabilities.property AS Property
| lookup kv_opsdb property as Property output contact
| lookup aws_accounts account_number AS Property OUTPUT account_name
| eval Property=if(isnull(account_name), Property, account_name)
| lookup workday.csv shortid as contact output l2
| sort - "Total Vulnerabilities"
| dedeup l2

Heres the result:
Vulnerabilities.signature Total Vulnerabilities Contact L2
CentOS 7 : bind (CESA-2019:1294) 129345 Jane Doe John Doe
Jane Doe John Doe
Jane Doe John Doe
Jane Doe John Doe
Jane Doe John Doe
Jane Doe John Doe
Jane Doe John Doe

and it goes on for a couple more hundred times. i just need the contact name and l2 showing once

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi payton_tayvion,
ay first probably it's a copy error, but at the end you have dedeup instead dedup.
Then are there differences using or not the sort command?
Then have you more than 50,000 results? if yes add 0 to the sort command ( | sort 0 l2 ).

Bye.
Giuseppe

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

A Four-Part Event Series: Full Stack Observability For the AI Era

As AI reshapes applications, infrastructure, and the way teams operate, the traditional boundaries of ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...