topic Re: using rex mode="sed" to remove strings surrounded by # characters in Getting Data In

using rex mode="sed" to remove strings surrounded by # characters

idjagger — Fri, 28 Feb 2020 13:06:19 GMT

Hi,

I have a series of log entries that are in the form

#4 MyApp\Framework\DB\Adapter\Pdo\Mysql->_query('SELECT `store_we...', array()) called at [vendor/myapp/framework/DB/Adapter/Pdo/Mysql.php:621]
#5 MyApp\Framework\DB\Adapter\Pdo\Mysql->query(MyApp\Framework\DB\Select#b8e969b2c2d6#, array()) called at [vendor/myapp/zendframework1/library/Zend/Db/Adapter/Abstract.php:737]
#6 Zend_Db_Adapter_Abstract->fetchAll(MyApp\Framework\DB\Select#b8e969b2c2d6#) called at [vendor/myapp/module-store/App/Config/Source/RuntimeConfigSource.php:87]

where the bits between # and # are the only differences between them.

I wanted to use sed to replace the bits between # and # with a common string so when I do a stats on them they all appear the same and I can get a nice count of how often the error occurred.

I've tried using

rex field=Message mode=sed "s/(Select\#[^\#]*\#)/Select/g"

What would I need to do to replace the portions between the # and # (in this case #b8e969b2c2d6#

Thanks,

Kind regards,

Ian

Re: using rex mode="sed" to remove strings surrounded by # characters

jpolvino — Fri, 28 Feb 2020 16:26:48 GMT

Here is one way to do it, using a Run Anywhere example:

| makeresults 
| eval raw="#4 MyApp\Framework\DB\Adapter\Pdo\Mysql->_query('SELECT `store_we...', array()) called at [vendor/myapp/framework/DB/Adapter/Pdo/Mysql.php:621]:::#5 MyApp\Framework\DB\Adapter\Pdo\Mysql->query(MyApp\Framework\DB\Select#b8e969b2c2d6#, array()) called at [vendor/myapp/zendframework1/library/Zend/Db/Adapter/Abstract.php:737]:::#6 Zend_Db_Adapter_Abstract->fetchAll(MyApp\Framework\DB\Select#b8e969b2c2d6#) called at [vendor/myapp/module-store/App/Config/Source/RuntimeConfigSource.php:87]"
| makemv delim=":::" raw
| mvexpand raw
| rename raw AS _raw
| rex field=_raw mode=sed "s/(#.+?#)([^#]+)(#.*)/\1StaticText\3/g"

What this does is break up eligible events into 3 parts, substituting the second part with StaticText.

You can see the regex here: Regex101

Re: using rex mode="sed" to remove strings surrounded by # characters

to4kawa — Fri, 28 Feb 2020 18:07:27 GMT

| rex mode=sed "s/Select#.*?#/Select/g"

just try it.