topic Re: replace is giving empty string in Splunk Search

Why is replace giving empty string?

YatMan — Tue, 30 May 2023 03:32:15 GMT

Sample event

{ durationMs: 83 properties: { request-id: 1c910793-8be4-4850-83d5-f360b4b05478 method: GET path: /scenarios/636d40506930b10b8f082f27 } }

I am trying to create a table of the counts by properties.path
I want to combine some of the rows into single path /scenarios/{id}
But my replace('properties.path') is giving empty value as seen in column values(path), please help me take a look why replace doesn't work here.

Re: replace is giving empty string

richgalloway — Sat, 27 May 2023 00:27:48 GMT

It would help if you posted the SPL as text rather than a screen shot so we can test with it.

The regex in the replace command doesn't match the data shown. It's looking for at least 15 letters or digits or any number of digits after the first slash, but the sample data has only 10 characters.

Re: replace is giving empty string

ITWhisperer — Sat, 27 May 2023 06:41:44 GMT

Here is a runanywhere example showing your replace working

| makeresults | fields - _time | eval _raw="{ \"durationMs\": 83, \"properties\": { \"request-id\": \"1c910793-8be4-4850-83d5-f360b4b05478\", \"method\": \"GET\", \"path\": \"/scenarios/636d40506930b10b8f082f27\" } }" | spath | eval path=replace('properties.path',"(\/[0-9a-zA-Z]{15,}|\/\d+)","/{id}") | stats values(path) by properties.path , properties.method

Which version of Splunk are you running?

Re: replace is giving empty string

YatMan — Tue, 30 May 2023 16:39:16 GMT

Your runanywhere example works, even on my side. But when I actually use it on my real event logs, the same problem occur. So looks the problem is not in replace, but how I feed the properties.url into replace. Thank you, I will do more troubleshooting on my side.

Re: replace is giving empty string

yuanliu — Tue, 30 May 2023 16:58:25 GMT

The most common string manipulation "failure" is caused by a field being multivalued. Any chance your data can give multivalued properties.path? Does your replace fail to render {id} with every properties.method or only some of them?

One easy test for multivaluedness can be

| eval path=mvmap('properties.path', replace('properties.path',"(\/[0-9a-zA-Z]{15,}|\/\d+)","/{id}")) | stats values(path) by properties.path , properties.method

Re: replace is giving empty string

YatMan — Tue, 30 May 2023 18:45:23 GMT

multivalue field as if 'properties.path' could contains more than one value? No, 'properties.path' always only have 1 value (1 string). And this field always exist in all logs.

However, your mvmap is able to make 'properties.path' work with replace(). Although I can't quite explain it, this is what I want to achieve. Thank you.

Re: replace is giving empty string

yuanliu — Wed, 31 May 2023 05:23:42 GMT

In that case, I do have another hypothesis. Is it possible that your source type uses both index-time extraction of JSON structure (INDEXED_EXTRACTIONS=JSON) and search time automatic extraction (KV_MODE=JSON)? A field can not only be multivalued and have several different values per event, but also be multivalued with identical values.

If every event has properties.path populated, this faux multivalue condition can most easily identified by looking at the fields column in smart mode or verbose mode. You will see that properties.path is populated in 200% of events. (As opposed to 100%.) If the fields is sparsely populated, you will need something like

base search properties.path=*

Hope this helps.

I remember reading warnings about KV_MODE and INDEX_EXTRACTIONS in Splunk docs, but cannot find examples in a quick search.