Knowledge Management

What is the least expensive way of removing multiple substrings from a string, and can I do it within a data model?

andrewtrobec
Motivator

Hello,

My objective is to clean three distinct substrings from a comma delimited string. Those substrings may all be present in the string, may not all be present in the string, or may not be present at all in the string. Their positions within the string can vary as well. Assuming values substring1, substring2, and substring3, here are some examples:

this,is,substring1,a,sentence,with,one
substring2,this,has,substring1,all,three,substring3
here,there,are,no,substrings
this,only,substring3,substring1,has,two

Ideally I would like to encorporate the logic within a data model, which limits me to eval or rex ( replace isn't possible). So far I can do it with rex mode=sed but I can't add it to a data model. Here is a run anywhere with my sed solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| rex mode=sed field=string "s/$/,/g"
| rex mode=sed field=string "s/substring1,//g"
| rex mode=sed field=string "s/substring2,//g"
| rex mode=sed field=string "s/substring3,//g"
| rex mode=sed field=string "s/.$//g"

The first and last sed commands are to add a comma to the end of the string to manage the case where a substring is positioned at the end, and to remove it again to clean up afterwards.

Are there any better solutions?

Thanks in advance, and best regards,

Andrew

Labels (1)
0 Karma
1 Solution

andrewtrobec
Motivator

OK i figured it out! I realized that replace can also be used as part of an eval. Here is the run anywhere solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| eval result = replace(string,"([\s,](substring1|substring2|substring3))|((substring1|substring2|substring3)[\s,])","")

Not sure whether there is a better way of doing it, so any suggestions are welcome!

View solution in original post

0 Karma

andrewtrobec
Motivator

OK i figured it out! I realized that replace can also be used as part of an eval. Here is the run anywhere solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| eval result = replace(string,"([\s,](substring1|substring2|substring3))|((substring1|substring2|substring3)[\s,])","")

Not sure whether there is a better way of doing it, so any suggestions are welcome!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...