Splunk Search

remove repetitive string in string

exocore123
Path Finder

manipulating strings, I had a post before regarding an array, but say I have a field that has value string1+string2+string1, how can I display this value as packaged(string1+string2)

EDIT:
I have a particular field in my logs that is displayed as field="string1+string2+string1" and when I use stats on this field to make a table, however, the string length can be very repetitive field="string1+string1+string1+string1..." or field="string1+string2+string2", so I want to be able to basically make any string with logically more than 1 value to be displayed as package(..) where the .. is the unique strings, so for field="string1+string1" can be packaged(string1) so does field="string1+string1+string1" and so forth, for
field="string1+string2+string2" or field="string1+string2" can be packaged(string1+string2)
Say each string represents different codes that are vastly different from one and another, so wildcard isnt an option.

0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

Try this -

| makeresults | eval myfield="asdw+wkas+asdw" 
| rename COMMENT as "The above creates test data"

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,"package(".mvjoin(mvdedup(myfield),"+").")")

with resulting output...

myfield
package(asdw+wkas)  

Now, since you are not retaining the info on how many of each were originally there, and since the field already had multiple values in it, you could forego the words "package()" around the altered field if you wanted.

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,mvjoin(mvdedup(myfield),"+"))

View solution in original post

DalJeanis
SplunkTrust
SplunkTrust

Try this -

| makeresults | eval myfield="asdw+wkas+asdw" 
| rename COMMENT as "The above creates test data"

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,"package(".mvjoin(mvdedup(myfield),"+").")")

with resulting output...

myfield
package(asdw+wkas)  

Now, since you are not retaining the info on how many of each were originally there, and since the field already had multiple values in it, you could forego the words "package()" around the altered field if you wanted.

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,mvjoin(mvdedup(myfield),"+"))

View solution in original post

exocore123
Path Finder

Works perfectly, one comment, say I have a string3 I do not want to add after the makemv, say string1+string2+string3, so after makemv it becomes string1 string2 string3, say I want to mvjoin only string1 and string2, is there a selective way to do this?

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

There are two easy ways, depending on how many of string3 you have.

You can use mvfilter ...

| makemv delim="+" myfield 
| eval myfield=mvfilter(NOT match(myfield,"idonotwantthis|orthis"))
| eval myfield=if(mvcount(myfield)=1, myfield, "package(".mvjoin(mvdedup(myfield),"+").")")

If the list of wanted values is shorter than the list of unwanted ones, then remove the NOT and put your desired strings connected by the vertical OR bars (|).

| eval myfield=mvfilter(match(myfield,"iwantthis|andthis|andthese"))

Or you can also use rex in sed mode ...

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)=1, myfield, "package(".mvjoin(mvdedup(myfield),"+").")")

| rename COMMENT as "The first sed gets rid of the undesired strings" 
| rex field=myfield mode=sed "s/idonotwantthis|orthis//g"

| rename COMMENT as  "The second sed gets rid of excess plusses at the front, middle or end of the string."
| rex field=myfield mode=sed "^\+|\+(?=\+)|\+$"

There are also some more complex ways if you have a bunch of these undesired strings to check for.

exocore123
Path Finder

I think I get the idea, thanks!

0 Karma

niketnilay
Legend

@exocore123, you might have to add sample values(if actual value is not possible please add anonymized string) for us to help you better.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

exocore123
Path Finder

@niketnilay idk if that helps but i tried to be more specific

0 Karma

niketnilay
Legend

@exocore123, by values I meant some sample data.

Does your strings in the field actually have plus sign (+) or any other delimiter between them?
Does your field actually have string1 or string2 in it? Or is this sample?

Or is your field value something like following?
field= "abcdefabc"
Where abc is string1 and def is string2.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

exocore123
Path Finder

The things I put are the literal sample data. The + sign is part of the string. It's literally field="asdw+wkas+asdw", not sure if that clarifies?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!