topic Re: Transforms not working trying to extract KV pairs from URL in Getting Data In

Transforms not working trying to extract KV pairs from URL

pdominicb — Wed, 13 May 2026 14:13:28 GMT

I have events with URLs, and the URLs contain parameters with KV values in them. Splunk auto extracts the KV pairs, but I want the transforms to apply a prefixed value to the key names from the pairs in the URL. For some reason my transforms refuse to work.

Here are a few sample events:

05-11-2026T14:08:01-05:00 LogLevel=INFO TraceId=69baf81038352c1e63cae59b98259742 Url=https://api.something.com/rsi/something/replacement/internal/something_internal_services/resources/org/p/ZMC31/01?hierarchy=false&includeInactive=true&test1=test2&test3=test4 DurationMillis=109 hierarchy=maybe includeInactive=false 05-11-2026T14:07:54-05:00 LogLevel=INFO TraceId=69baf80a0a882f20e6c944e61c5109a2 Url=https://api.something.com/rsi/something/replacement/internal/something_internal_services/resources/org/p/3UC31/01?hierarchy=false&includeInactive=true&test1=test2&test3=test4 DurationMillis=106 hierarchy=maybe includeInactive=false

I would like the extracted results from the URL to look like:

QP_hierarchy=false QP_includeInactive=true QP_test1=test2 QP_test3=test4

Here are my props and transforms:

[app_transaction] LINE_BREAKER = ([\r\n]+ MAX_TIMESTAMP_LOOKAHEAD = 30 SHOULD_LINEMERGE = False TIME_PREFIX = ^ TRANSFORMS-extract = app_transaction_params TRUNCATE = 99999 [app_transaction_params] FORMAT = QP_$1::$2 MV_ADD = false REGEX = (?:\?|&)(\w+)=([^&\s]+) REPEAT_MATCH = true SOURCE_KEY = _raw

The regex seems to work in regex101.com and extracts the groups correctly. The props is line breaking correctly. Any ideas whats wrong with my transforms here?

Re: Transforms not working trying to extract KV pairs from URL

inventsekar — Wed, 13 May 2026 15:15:39 GMT

Hi @pdominicb

1) may i know where did you configure the props and transforms?
is it on indexers or search heads or heavy forwarders or UF's?

2) after updating the props and transforms, did you restart the splunk service?

----------------------------------------------------------------------------------------------
If this post or any post addressed your question, could you pls:
Give it karma to show appreciation

PS - As of May 2026, my Karma Given is 2312 and my Karma Received is 497, lets revamp the Karma Culture!
Thanks and best regards, Sekar
--------------------------------------------------------------------------------------------

Re: Transforms not working trying to extract KV pairs from URL

pdominicb — Wed, 13 May 2026 15:19:35 GMT

Oh sorry, I should have added that context in my post.

The props and transforms are both on the indexers, and yes Splunk has been restarted. Although I believe when props and transforms are pushed to indexers the deploy will do a reload, correct?

Re: Transforms not working trying to extract KV pairs from URL

inventsekar — Wed, 13 May 2026 15:26:53 GMT

to understand whether the props and transforms are being applied or not, you can use the btool command

/opt/splunk/bin/splunk btool props list <sourcetype> --debug

/opt/splunk/bin/splunk btool transforms list <transform_name> --debug

Re: Transforms not working trying to extract KV pairs from URL

pdominicb — Wed, 13 May 2026 15:30:27 GMT

It says they are, but its not applying the prefix in Splunk.

[splunk@myhost ~]$ /opt/splunk/bin/splunk btool props list app_transaction --debug | grep local /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf [app_transaction] /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf LINE_BREAKER = ([\r\n]+ /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf MAX_TIMESTAMP_LOOKAHEAD = 30 /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf SHOULD_LINEMERGE = False /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf TIME_PREFIX = ^ /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf TRANSFORMS-extract = app_transaction_params /opt/splunk/etc/apps/ehi_dev_standard_props/local/props.conf TRUNCATE = 99999 [splunk@myhost ~]$ /opt/splunk/bin/splunk btool transforms list app_transaction_params --debug | grep local /opt/splunk/etc/apps/ehi_dev_standard_props/local/transforms.conf [app_transaction_params] /opt/splunk/etc/apps/ehi_dev_standard_props/local/transforms.conf FORMAT = QP_$1::$2 /opt/splunk/etc/apps/ehi_dev_standard_props/local/transforms.conf MV_ADD = false /opt/splunk/etc/apps/ehi_dev_standard_props/local/transforms.conf REGEX = (?:\?|&)(\w+)=([^&\s]+) /opt/splunk/etc/apps/ehi_dev_standard_props/local/transforms.conf REPEAT_MATCH = true /opt/splunk/etc/apps/ehi_dev_standard_props/local/transforms.conf SOURCE_KEY = _raw

Re: Transforms not working trying to extract KV pairs from URL

livehybrid — Wed, 13 May 2026 23:33:39 GMT

Hi @pdominicb

This configuration would create index-time extractions not search-time, is that intentional? Therefore changes you make will not apply to previously ingested data.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Transforms not working trying to extract KV pairs from URL

pdominicb — Thu, 14 May 2026 12:48:10 GMT

That is the intention and I'm aware that it would not apply to previously indexed data. However, if you think this would work better (or at all) as a search time extraction I would be ok with that as a solution. It was my impression that this would not work at search time and had to be on indexers. Do you think I should move this to the search heads?

EDIT: I just tried this on a single instance deployment and am still not seeing the field names show up prefixed with "QP_" like QP_hierarchy. So it just seems like there is something wrong in my transforms, or my props isn't actually calling the transforms for some reason.

Re: Transforms not working trying to extract KV pairs from URL

PickleRick — Fri, 15 May 2026 15:57:03 GMT

Adding to @livehybrid 's comment - as a rule of thumb, search time extractions are "better". Unless you have a really border case, it's better to create a search time extraction (and maybe accelerate the search in one of possible ways if you need it) than to create index time extractions.

Anyway, my first guess would be that you might by any chance using indexed extractions.

Secondly, I can never get my head around when you need the WRITE_META setting and when you don't.

But still I'd just go for search time extraction.

Re: Transforms not working trying to extract KV pairs from URL

isoutamo — Fri, 15 May 2026 23:08:41 GMT

Splunk has created some (or actually quite many) parsing tokens for different http URLs. You should just look those from $SPLUNK_HOME/etc/system/default/props+transfoms.conf files. Basically you could utilize those generic tokenizations in your own definitions too. Those are parsing URLs quite well.

Re: Transforms not working trying to extract KV pairs from URL

pdominicb — Mon, 18 May 2026 14:40:50 GMT

I see "simple_url" and "url" in the transforms.conf spec. But they look like they only extract the url, domain, uri, and a few other things. But they don create extracted KV pairs from a url parameter string.

Let me know if I'm missing any.

Re: Transforms not working trying to extract KV pairs from URL

isoutamo — Fri, 22 May 2026 14:11:37 GMT

When you are looking access_combined from .../etc/system/default/props.conf you see that it's using transforms REPORT-access = access-extractions
Then when you looking it, you see lot of recursive definitions where those parts are divided.
Those are after this comments

######## access-extractions helpers start ########
# make sure to handle escaped quotes (\") inside the URI

[access-extractions] # matches access-common or access-combined apache logging formats # Extracts: clientip, clientport, ident, user, req_time, method, uri, root, file, uri_domain, uri_query, version, status, bytes, referer_url, referer_domain, referer_proto, useragent, cookie, other (remaining chars) # Note: referer is misspelled in purpose because that is the "official" spelling for "HTTP referer" REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]](?:\s++"(?<referer>[[bc_domain:referer_]]?+[^"]*+)"(?:\s++[[qstring:useragent]](?:\s++[[qstring:cookie]])?+)?+)?[[all:other]]

all those [[<type>:<field name>]] are tokens which get values.
Just look deeper which each <type> is, then maybe you need to look again what those contains etc.

e.g access-request is

[access-request] # very relaxed regex for extracting fields from the request REGEX = "\s*+[[reqstr:method]]?(?:\s++[[bc_uri]](?:\s++[[reqstr:version]])*)?\s*+"

I contains e.g. bc_uri this is defined as

[bc_uri] # backwards compatible uri regex # uri = path optionally followed by query [/this/path/file.js?query=part&other=var] # path = root part followed by file [/root/part/file.part] # Extracts: uri, uri_path, root, file, uri_query, uri_domain (optional if in proxy mode) REGEX = (?<uri>[[bc_domain:uri_]]?+(?<uri_path>[[uri_root]]?[[uri_seg]]*(?<file>[^\s\?/]+)?)(?:\?(?<uri_query>[^\s]*))?)

And so on.