Splunk Search

Replace multiple text strings with different values

donB
Loves-to-Learn Lots

i have to replace multiple text strings with different values. e.g.

Log Statement:- 

"Hello, this is sample url for employees, /company/employee/1 and there is also a sample url for departments like /company/name/department/a1"

I have to replace any url's of the format "/company/employee/*" with /company/employee{id} ,

and url's of format "/company/name/department/*"  with  "/company/name/department/{deptId}"

and url's for format "/company/notfound/404" with "/company/notfound/{status} 

and so on..

so output will look like below.

"Hello, this is sample url for employees, /company/employee/{id} and there is also a sample url for departments like /company/name/department/a1 or /company/name/department/{deptId}"

Currently i am using 3 nested replace statements for above example to solve this issue because i don't know how many patterns would exist in the statement so i check all 3 patterns. Nested replace seems like slow and also giving errors like below.

  • has exceeded configured match_limit, consider raising the value in limits.conf.

Also my nested replace statements are increasing as i am adding more url formats. this is exactly how i am forming the regex.

 

| eval apiPath = replace
(replace(replace(replace(replace(replace(replace(replace(replace(replace(
trimmedUrl, "\/[0-9]{1}[0-9a-z]+\/", "/{id}/"
), "\/[a-z]{1,2}[0-9]{1}[0-9a-z]+\/", "/{id}/"
), "\?.*", "?<my-filters>"
), "\/P-[0-9]+", "/P-{id}"
), "\/car\/[a-z]+\/.*", "/car/{carType}/{id}"
), "\b[0-9a-f]{8}\b-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-\b[0-9a-f]{12}\b", "{uuid}"
), "\/user\/[a-z0-9+\._-]+(@|%2540)[a-z\.-]+\.[a-z]+$", "/user/{email}"
), "\/(jobs|xls)\/[a-z0-9]+", "/\1/{id}"
), "\/org\/.*\/enroll\/status", "/org/{id}/enroll/status"
), "\/car\/[a-zA-Z]+\/[0-9]+$", "/car/{carType}/{id}"
)

i want to remove the error and execute the search faster. please suggest couple of different options. thank you

Labels (6)
0 Karma

ITWhisperer
Ultra Champion

Why not try doing the replaces serially instead of nesting them?

0 Karma