Splunk Search
Highlighted

Why is the wrong value being extracted when using this regular expression?

Communicator

Hi,

I am using a regular expression to extract the word that follows the string result of raw output. For endpoint 1 the captured value is "s"(incorrect) and for endpoint 2 the captured value is "OK" (correct).

Using Splunk Enterprise 6.5.1 build f74036626f0c, and the regex was generated using RegexBuddy (language PCRE2 10.21 - closest to splunk, and here the correct value is highlighted in both cases).

My inputs, props, transforms and raw output below. Would like some help on this, as i fail to understand from where "s" is captured.

inputs.conf

[rest://test]
source = test
auth_type = none
endpoint = http://localhost:8130/test/v1/statuscheck
http_method = GET
index = main
index_error_response_codes = 0
polling_interval = 60
request_timeout = 50
response_type = xml
sequential_mode = 0
sourcetype = url
streaming_request = 0

[rest://test2]
source = test2
auth_type = none
endpoint = http://localhost:8131/test/v1/statuscheck
http_method = GET
index = main
index_error_response_codes = 0
polling_interval = 60
request_timeout = 50
response_type = xml
sequential_mode = 0
sourcetype = url
streaming_request = 0

props.conf

[url]
category = Custom
pulldown_type = 1
disabled = false
TRANSFORMS-url = url_transformation

transforms.conf

[url_transformation]
REGEX = ^.+<result>(?<url_status>\w+).+
FORMAT = url_status::$1
WRITE_META = true

Endpoint 1 raw output :

curl http://localhost:8130/test/v1/statuscheck
<status>
        <result>OK</result>
        <resources/>

Endpoint 2 raw output :

curl http://localhost:8131/test/v1/statuscheck
<status>
<result>OK</result>
<resources><resource name="..." status="OK" /><resource name="..." status="OK" /></resources>
</status>
0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Builder

try this regex in your transforms.conf

REGEX = <result>(?<url_status>\w+).+

Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Super Champion

using regex101.com , using .+<result>(?<url_status>\w+).+ seemed to work for both, the carrot at the beginning was throwing something off.

0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Communicator

using regex101.com i have the same result as with my regex. The correct value is selected, but when this is applied to transforms.conf , the result value for the field is still"s" . Tested your regex after clearing my test environment data , and i get the same result... the value is "s" for endpoint 1.

0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Communicator

Forgot to mention that I am using Heavy Forwarders and the inputs,props and transforms files sit on the forwarder and not indexer.

0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

SplunkTrust
SplunkTrust

How about this

 REGEX = \<result\>(?<url_status>[^\<]+)\<\/result\>
0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Communicator

Works in search , but when put in transforms.conf and clearing my test environment data ,i get the same result... the value is "s" for endpoint 1.

0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

SplunkTrust
SplunkTrust

So, you're keeping the configuration on indexer/heavy forwarder and restarting Splunk on that host?

0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Communicator

configuration is kept on heavy forwarder, cleaning data from indexer and restarting indexer to have fresh data.

0 Karma
Highlighted

Re: Why is the wrong value being extracted when using this regular expression?

Esteemed Legend

Is this how you are testing?

1: Remove ALL the existing (bad) data from the indexers by running a search that pulls it in and piping it to the delete command on the search head.
2: Update your configurations on the Heavy Forwarder and restart splunk there.
3: Run a search and be sure that the data is "still gone".
4: Forward the data in.
5: Run the search again and see new data.

This is important because the configuration changes only effect NEWLY FORWARDED events.

0 Karma