Splunk Search

Remove Alike Values in a Field

whitefang1726
Path Finder

Hello Splunk Answers, How can I remove this duplicate line? See sample below:

From: 

row1     row2       row3
1.1.1.1  XXX         alpha.splunk.com
                                 alpha
2.2.2.2  YYY         beta.splunk.com
                                 BETA
3.3.3.3  ZZZ        delta.splunkanswers.com
                                 delta
4.4.4.4  AAA        abcdefgh

to: 
row1     row2       row3
1.1.1.1  XXX         alpha.splunk.com
2.2.2.2  YYY         beta.splunk.com
3.3.3.3  ZZZ        delta.splunkanswers.com
4.4.4.4  AAA        abcdefgh

Thanks!

Labels (1)
0 Karma
1 Solution

johnhuang
Motivator

If they are multivalue fields and you want the first:

| eval row3=mvindex(row3, 0)

 

View solution in original post

0 Karma

johnhuang
Motivator

If they are multivalue fields and you want the first:

| eval row3=mvindex(row3, 0)

 

0 Karma

whitefang1726
Path Finder

This works, thanks!

0 Karma

whitefang1726
Path Finder

Okay, here's another example. 

from: 
field1 field2 field3 field4
row1 1.1.1.1 XXX alpha.splunk.com
alpha
row2 2.2.2.2 YYY beta.splunk.com
BETA
row3 3.3.3.3 ZZZ delta.splunkanswers.com
delta
row4 4.4.4.4 AAA abcdefgh

to:
field1 field2 field3 field4
row1 1.1.1.1 XXX alpha.splunk.com
row2 2.2.2.2 YYY beta.splunk.com
row3 3.3.3.3 ZZZ delta.splunkanswers.com
row4 4.4.4.4 AAA abcdefgh

or:

field2 field3 field4
1.1.1.1 XXX alpha.splunk.com
2.2.2.2 YYY beta.splunk.com
3.3.3.3 ZZZ delta.splunkanswers.com
4.4.4.4 AAA abcdefgh

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Sorry, that doesn't really help. Can you provide (sanitised) examples of your raw events and the searches you are currently using to get your current results?

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It is not clear what these "lines" are - are they all in a single field called row3 with linebreaks? are they multivalues? have they been created by stats values/list functions, or transaction commands?

0 Karma

whitefang1726
Path Finder

This is a sample output using stats values. Sorry, I give a wrong example. Here's the current one.

from:
field1     field2     field3     field4
row1     1.1.1.1     XXX     alpha.splunk.com
                                                   alpha
row2     2.2.2.2     YYY     beta.splunk.com
                                                   BETA
row3     3.3.3.3     ZZZ     delta.splunkanswers.com
                                                 delta
row4     4.4.4.4     AAA     abcdefgh

To: 
field1     field2     field3     field4
row1     1.1.1.1     XXX     alpha.splunk.com
row2     2.2.2.2     YYY     beta.splunk.com
row3     3.3.3.3     ZZZ     delta.splunkanswers.com
row4     4.4.4.4     AAA     abcdefgh

So, for field4, alpha.splunk.com and alpha is just one value (multi-value), beta and beta.splunk.com is one value (multi-value), so on...
I want to remove the redundant names (hostname only and host with FQDN) as long it same value (example: alpha and alpha.splunk.com)


Tags (1)
0 Karma
Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...