Reporting

How to find duplicate values inside a single field?

v610469
New Member

For data from DNS that looks like these examples:
www.abc.com.www.bca.com
www.abc.net.www.bca.net

How can I report that .com or .net appears more than once in the field? I would stat this an push it into a count by client after finding the values that match.

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

if you have a single value per field, you can create a new field and trim the domain using a regex.

    __mysearch>__ | rex field=mydomainfield "(?<start_domain>.*)(?<end_domain>\.\w+)$" | table mydomainfield start_domain end_domain

the do add a

      | stats count by start_domain

if you have mutlivalue per field, you have to start by splitting your field.
see this article
http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Parsemultivaluefields

View solution in original post

0 Karma

dcarmack_splunk
Splunk Employee
Splunk Employee

try this

| rex field=domain max_match=10 "(?<top_level_domain>\b(?:com|net|edu|org)\b)" | stats count by domain top_level_domain | where count > 1 

yannK
Splunk Employee
Splunk Employee

if you have a single value per field, you can create a new field and trim the domain using a regex.

    __mysearch>__ | rex field=mydomainfield "(?<start_domain>.*)(?<end_domain>\.\w+)$" | table mydomainfield start_domain end_domain

the do add a

      | stats count by start_domain

if you have mutlivalue per field, you have to start by splitting your field.
see this article
http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Parsemultivaluefields

0 Karma

v610469
New Member

Yes will be multivalued field. Where we want to report clients who for some reason append additional domains in a DNS query, which will fail except in rare cases.

I did get around this by using .net. OR .com. the trailing dot picks up there is more to the domain.

Paula

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...