Knowledge Management

What is the least expensive way of removing multiple substrings from a string, and can I do it within a data model?

andrewtrobec
Motivator

Hello,

My objective is to clean three distinct substrings from a comma delimited string. Those substrings may all be present in the string, may not all be present in the string, or may not be present at all in the string. Their positions within the string can vary as well. Assuming values substring1, substring2, and substring3, here are some examples:

this,is,substring1,a,sentence,with,one
substring2,this,has,substring1,all,three,substring3
here,there,are,no,substrings
this,only,substring3,substring1,has,two

Ideally I would like to encorporate the logic within a data model, which limits me to eval or rex ( replace isn't possible). So far I can do it with rex mode=sed but I can't add it to a data model. Here is a run anywhere with my sed solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| rex mode=sed field=string "s/$/,/g"
| rex mode=sed field=string "s/substring1,//g"
| rex mode=sed field=string "s/substring2,//g"
| rex mode=sed field=string "s/substring3,//g"
| rex mode=sed field=string "s/.$//g"

The first and last sed commands are to add a comma to the end of the string to manage the case where a substring is positioned at the end, and to remove it again to clean up afterwards.

Are there any better solutions?

Thanks in advance, and best regards,

Andrew

Labels (1)
0 Karma
1 Solution

andrewtrobec
Motivator

OK i figured it out! I realized that replace can also be used as part of an eval. Here is the run anywhere solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| eval result = replace(string,"([\s,](substring1|substring2|substring3))|((substring1|substring2|substring3)[\s,])","")

Not sure whether there is a better way of doing it, so any suggestions are welcome!

View solution in original post

0 Karma

andrewtrobec
Motivator

OK i figured it out! I realized that replace can also be used as part of an eval. Here is the run anywhere solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| eval result = replace(string,"([\s,](substring1|substring2|substring3))|((substring1|substring2|substring3)[\s,])","")

Not sure whether there is a better way of doing it, so any suggestions are welcome!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...