Splunk Search

Remove Alike Values in a Field

whitefang1726
Path Finder

Hello Splunk Answers, How can I remove this duplicate line? See sample below:

From: 

row1     row2       row3
1.1.1.1  XXX         alpha.splunk.com
                                 alpha
2.2.2.2  YYY         beta.splunk.com
                                 BETA
3.3.3.3  ZZZ        delta.splunkanswers.com
                                 delta
4.4.4.4  AAA        abcdefgh

to: 
row1     row2       row3
1.1.1.1  XXX         alpha.splunk.com
2.2.2.2  YYY         beta.splunk.com
3.3.3.3  ZZZ        delta.splunkanswers.com
4.4.4.4  AAA        abcdefgh

Thanks!

Labels (1)
0 Karma
1 Solution

johnhuang
Motivator

If they are multivalue fields and you want the first:

| eval row3=mvindex(row3, 0)

 

View solution in original post

0 Karma

johnhuang
Motivator

If they are multivalue fields and you want the first:

| eval row3=mvindex(row3, 0)

 

0 Karma

whitefang1726
Path Finder

This works, thanks!

0 Karma

whitefang1726
Path Finder

Okay, here's another example. 

from: 
field1 field2 field3 field4
row1 1.1.1.1 XXX alpha.splunk.com
alpha
row2 2.2.2.2 YYY beta.splunk.com
BETA
row3 3.3.3.3 ZZZ delta.splunkanswers.com
delta
row4 4.4.4.4 AAA abcdefgh

to:
field1 field2 field3 field4
row1 1.1.1.1 XXX alpha.splunk.com
row2 2.2.2.2 YYY beta.splunk.com
row3 3.3.3.3 ZZZ delta.splunkanswers.com
row4 4.4.4.4 AAA abcdefgh

or:

field2 field3 field4
1.1.1.1 XXX alpha.splunk.com
2.2.2.2 YYY beta.splunk.com
3.3.3.3 ZZZ delta.splunkanswers.com
4.4.4.4 AAA abcdefgh

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Sorry, that doesn't really help. Can you provide (sanitised) examples of your raw events and the searches you are currently using to get your current results?

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It is not clear what these "lines" are - are they all in a single field called row3 with linebreaks? are they multivalues? have they been created by stats values/list functions, or transaction commands?

0 Karma

whitefang1726
Path Finder

This is a sample output using stats values. Sorry, I give a wrong example. Here's the current one.

from:
field1     field2     field3     field4
row1     1.1.1.1     XXX     alpha.splunk.com
                                                   alpha
row2     2.2.2.2     YYY     beta.splunk.com
                                                   BETA
row3     3.3.3.3     ZZZ     delta.splunkanswers.com
                                                 delta
row4     4.4.4.4     AAA     abcdefgh

To: 
field1     field2     field3     field4
row1     1.1.1.1     XXX     alpha.splunk.com
row2     2.2.2.2     YYY     beta.splunk.com
row3     3.3.3.3     ZZZ     delta.splunkanswers.com
row4     4.4.4.4     AAA     abcdefgh

So, for field4, alpha.splunk.com and alpha is just one value (multi-value), beta and beta.splunk.com is one value (multi-value), so on...
I want to remove the redundant names (hostname only and host with FQDN) as long it same value (example: alpha and alpha.splunk.com)


Tags (1)
0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...