Knowledge Management

What is the least expensive way of removing multiple substrings from a string, and can I do it within a data model?

andrewtrobec
Motivator

Hello,

My objective is to clean three distinct substrings from a comma delimited string. Those substrings may all be present in the string, may not all be present in the string, or may not be present at all in the string. Their positions within the string can vary as well. Assuming values substring1, substring2, and substring3, here are some examples:

this,is,substring1,a,sentence,with,one
substring2,this,has,substring1,all,three,substring3
here,there,are,no,substrings
this,only,substring3,substring1,has,two

Ideally I would like to encorporate the logic within a data model, which limits me to eval or rex ( replace isn't possible). So far I can do it with rex mode=sed but I can't add it to a data model. Here is a run anywhere with my sed solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| rex mode=sed field=string "s/$/,/g"
| rex mode=sed field=string "s/substring1,//g"
| rex mode=sed field=string "s/substring2,//g"
| rex mode=sed field=string "s/substring3,//g"
| rex mode=sed field=string "s/.$//g"

The first and last sed commands are to add a comma to the end of the string to manage the case where a substring is positioned at the end, and to remove it again to clean up afterwards.

Are there any better solutions?

Thanks in advance, and best regards,

Andrew

Labels (1)
0 Karma
1 Solution

andrewtrobec
Motivator

OK i figured it out! I realized that replace can also be used as part of an eval. Here is the run anywhere solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| eval result = replace(string,"([\s,](substring1|substring2|substring3))|((substring1|substring2|substring3)[\s,])","")

Not sure whether there is a better way of doing it, so any suggestions are welcome!

View solution in original post

0 Karma

andrewtrobec
Motivator

OK i figured it out! I realized that replace can also be used as part of an eval. Here is the run anywhere solution:

| makeresults | eval string="this,is,substring1,a,sentence,with,one"
| append [ | makeresults | eval string="substring2,this,has,substring1,all,three,substring3" ]
| append [ | makeresults | eval string="here,there,are,no,substrings" ]
| append [ | makeresults | eval string="this,only,substring3,substring1,has,two" ]
| table string
| eval result = replace(string,"([\s,](substring1|substring2|substring3))|((substring1|substring2|substring3)[\s,])","")

Not sure whether there is a better way of doing it, so any suggestions are welcome!

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...