Splunk Search

remove repetitive string in string

exocore123
Path Finder

manipulating strings, I had a post before regarding an array, but say I have a field that has value string1+string2+string1, how can I display this value as packaged(string1+string2)

EDIT:
I have a particular field in my logs that is displayed as field="string1+string2+string1" and when I use stats on this field to make a table, however, the string length can be very repetitive field="string1+string1+string1+string1..." or field="string1+string2+string2", so I want to be able to basically make any string with logically more than 1 value to be displayed as package(..) where the .. is the unique strings, so for field="string1+string1" can be packaged(string1) so does field="string1+string1+string1" and so forth, for
field="string1+string2+string2" or field="string1+string2" can be packaged(string1+string2)
Say each string represents different codes that are vastly different from one and another, so wildcard isnt an option.

0 Karma
1 Solution

DalJeanis
Legend

Try this -

| makeresults | eval myfield="asdw+wkas+asdw" 
| rename COMMENT as "The above creates test data"

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,"package(".mvjoin(mvdedup(myfield),"+").")")

with resulting output...

myfield
package(asdw+wkas)  

Now, since you are not retaining the info on how many of each were originally there, and since the field already had multiple values in it, you could forego the words "package()" around the altered field if you wanted.

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,mvjoin(mvdedup(myfield),"+"))

View solution in original post

DalJeanis
Legend

Try this -

| makeresults | eval myfield="asdw+wkas+asdw" 
| rename COMMENT as "The above creates test data"

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,"package(".mvjoin(mvdedup(myfield),"+").")")

with resulting output...

myfield
package(asdw+wkas)  

Now, since you are not retaining the info on how many of each were originally there, and since the field already had multiple values in it, you could forego the words "package()" around the altered field if you wanted.

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)<=1,myfield,mvjoin(mvdedup(myfield),"+"))

exocore123
Path Finder

Works perfectly, one comment, say I have a string3 I do not want to add after the makemv, say string1+string2+string3, so after makemv it becomes string1 string2 string3, say I want to mvjoin only string1 and string2, is there a selective way to do this?

0 Karma

DalJeanis
Legend

There are two easy ways, depending on how many of string3 you have.

You can use mvfilter ...

| makemv delim="+" myfield 
| eval myfield=mvfilter(NOT match(myfield,"idonotwantthis|orthis"))
| eval myfield=if(mvcount(myfield)=1, myfield, "package(".mvjoin(mvdedup(myfield),"+").")")

If the list of wanted values is shorter than the list of unwanted ones, then remove the NOT and put your desired strings connected by the vertical OR bars (|).

| eval myfield=mvfilter(match(myfield,"iwantthis|andthis|andthese"))

Or you can also use rex in sed mode ...

| makemv delim="+" myfield 
| eval myfield=if(mvcount(myfield)=1, myfield, "package(".mvjoin(mvdedup(myfield),"+").")")

| rename COMMENT as "The first sed gets rid of the undesired strings" 
| rex field=myfield mode=sed "s/idonotwantthis|orthis//g"

| rename COMMENT as  "The second sed gets rid of excess plusses at the front, middle or end of the string."
| rex field=myfield mode=sed "^\+|\+(?=\+)|\+$"

There are also some more complex ways if you have a bunch of these undesired strings to check for.

exocore123
Path Finder

I think I get the idea, thanks!

0 Karma

niketn
Legend

@exocore123, you might have to add sample values(if actual value is not possible please add anonymized string) for us to help you better.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

exocore123
Path Finder

@niketnilay idk if that helps but i tried to be more specific

0 Karma

niketn
Legend

@exocore123, by values I meant some sample data.

Does your strings in the field actually have plus sign (+) or any other delimiter between them?
Does your field actually have string1 or string2 in it? Or is this sample?

Or is your field value something like following?
field= "abcdefabc"
Where abc is string1 and def is string2.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

exocore123
Path Finder

The things I put are the literal sample data. The + sign is part of the string. It's literally field="asdw+wkas+asdw", not sure if that clarifies?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...