Splunk Search

How to use SED to remove optional fields?

singlinet
Engager

We have api requests that I want to create statistics by the request but to do this I need to remove variable identifiers and any parameters.

For example, with the following requestpatterns

POST /api-work-order/v1/work-orders/10611946/labours-reporting/2004131 HTTP/1.1
GET /api-work-order/v1/work-orders/10611946/labours-reporting HTTP/1.1
PUT /api-work-order/v1/work-orders/10611946 HTTP/1.1
GET /api-work-order/v1/work-orders HTTP/1.1

I need to replace the identifiers to extract:

POST /api-work-order/v1/work-orders/{id}/labours-reporting/{id}
GET /api-work-order/v1/work-orders/{id}/labours-reporting
PUT /api-work-order/v1/work-orders/{id}
GET /api-work-order/v1/work-orders 

 

Labels (1)
0 Karma
1 Solution

singlinet
Engager

Thanks for the suggestion.

I have resolved the issue by defining  the full string as field Endpoint and then doing multiple passes to strip the various numeric identifiers.

rex field=Endpoint mode=sed "s/(.*)(\/|$)\d{2,}$/\1\2<id>/g"
| rex field=Endpoint mode=sed "s/(.*)(\/)\d{2,}(\/.*$)/\1\2<id>\3/g"
| stats count by Endpoint


View solution in original post

yuanliu
SplunkTrust
SplunkTrust

SPL provides several methods to do regex replacement.  One method is rex mode=sed, the other replace.  Let's use rex since you are already considering sed.

| rex mode=sed "s/(\/api-work-order\/v1\/work-orders\/)\d+(.*)/\1{id}\2/"

 

Tags (1)
0 Karma

singlinet
Engager

Thanks for the suggestion.

I have resolved the issue by defining  the full string as field Endpoint and then doing multiple passes to strip the various numeric identifiers.

rex field=Endpoint mode=sed "s/(.*)(\/|$)\d{2,}$/\1\2<id>/g"
| rex field=Endpoint mode=sed "s/(.*)(\/)\d{2,}(\/.*$)/\1\2<id>\3/g"
| stats count by Endpoint


Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...