Splunk Search

Remove Alike Values in a Field

whitefang1726
Path Finder

Hello Splunk Answers, How can I remove this duplicate line? See sample below:

From: 

row1     row2       row3
1.1.1.1  XXX         alpha.splunk.com
                                 alpha
2.2.2.2  YYY         beta.splunk.com
                                 BETA
3.3.3.3  ZZZ        delta.splunkanswers.com
                                 delta
4.4.4.4  AAA        abcdefgh

to: 
row1     row2       row3
1.1.1.1  XXX         alpha.splunk.com
2.2.2.2  YYY         beta.splunk.com
3.3.3.3  ZZZ        delta.splunkanswers.com
4.4.4.4  AAA        abcdefgh

Thanks!

Labels (1)
0 Karma
1 Solution

johnhuang
Motivator

If they are multivalue fields and you want the first:

| eval row3=mvindex(row3, 0)

 

View solution in original post

0 Karma

johnhuang
Motivator

If they are multivalue fields and you want the first:

| eval row3=mvindex(row3, 0)

 

0 Karma

whitefang1726
Path Finder

This works, thanks!

0 Karma

whitefang1726
Path Finder

Okay, here's another example. 

from: 
field1 field2 field3 field4
row1 1.1.1.1 XXX alpha.splunk.com
alpha
row2 2.2.2.2 YYY beta.splunk.com
BETA
row3 3.3.3.3 ZZZ delta.splunkanswers.com
delta
row4 4.4.4.4 AAA abcdefgh

to:
field1 field2 field3 field4
row1 1.1.1.1 XXX alpha.splunk.com
row2 2.2.2.2 YYY beta.splunk.com
row3 3.3.3.3 ZZZ delta.splunkanswers.com
row4 4.4.4.4 AAA abcdefgh

or:

field2 field3 field4
1.1.1.1 XXX alpha.splunk.com
2.2.2.2 YYY beta.splunk.com
3.3.3.3 ZZZ delta.splunkanswers.com
4.4.4.4 AAA abcdefgh

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Sorry, that doesn't really help. Can you provide (sanitised) examples of your raw events and the searches you are currently using to get your current results?

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It is not clear what these "lines" are - are they all in a single field called row3 with linebreaks? are they multivalues? have they been created by stats values/list functions, or transaction commands?

0 Karma

whitefang1726
Path Finder

This is a sample output using stats values. Sorry, I give a wrong example. Here's the current one.

from:
field1     field2     field3     field4
row1     1.1.1.1     XXX     alpha.splunk.com
                                                   alpha
row2     2.2.2.2     YYY     beta.splunk.com
                                                   BETA
row3     3.3.3.3     ZZZ     delta.splunkanswers.com
                                                 delta
row4     4.4.4.4     AAA     abcdefgh

To: 
field1     field2     field3     field4
row1     1.1.1.1     XXX     alpha.splunk.com
row2     2.2.2.2     YYY     beta.splunk.com
row3     3.3.3.3     ZZZ     delta.splunkanswers.com
row4     4.4.4.4     AAA     abcdefgh

So, for field4, alpha.splunk.com and alpha is just one value (multi-value), beta and beta.splunk.com is one value (multi-value), so on...
I want to remove the redundant names (hostname only and host with FQDN) as long it same value (example: alpha and alpha.splunk.com)


Tags (1)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Asynchronous Forwarding Explained

Splunk asynchronous forwarding is often misunderstood as simply setting autoLBVolume. That is not quite right. ...

55 Days to Go: Secure Your Seat at Splunk University in Denver

Your .conf26 Experience Starts Before Opening Keynote  If Denver is known for its mile-high elevation, Splunk ...

(re)Introducing the Splunk Community Champions + 2026 – 2027 Splunk MVPs ...

This program exists as a channel to empower and recognize Splunk advocates and help supercharge initiatives to ...