extract fields from rex _raw

[oustinov]

trying to extract a fields from logfile's text (have both examples in logfile):

search sourcetype=apache "/apps/public/client1/local/" | "rex field=_raw "/apps/public/(?<client>\w+)/(?<region>\w+)/"" - works perfect, i see a fields "client" and "region" with correct client names

search sourcetype=apache "/apps/public/v4/client1/local/" | "rex field=_raw "/apps/public/v4/(?<client>\w+)/(?<region>\w+)/"" - does not work - no fields "client" and "region"

why and what is wrong?

[elliotproebstel · Accepted solution]

Give this a shot:

search sourcetype=apache "/apps/public/client1/local/" OR "/apps/public/v4/client1/local/"
| rex "\/apps\/public(\/v4)?\/(?<client>[^\/]+)\/(?<region>[^\/]+)"

This should ignore v4 if present but still allow it to work on URLs that do not contain v4. If you need it more generic, please share some examples where it fails, and I can help you iterate.

[oustinov]

Exact your rex does not work, but this:

rex field=_raw "\/apps\/public(\/v4)?\/(?<client>\w+)\/(?<region>\w+)" 

works perfect!
Many thanks!!!

[p_gurav]

Can you try below regex:

\/apps\/public\/*\w*\d*\/(?<client>\w+)\/(?<region>\w+)\/

[oustinov]

Hi,
this does not work, please see answer above - this works 100%
thanks!

[oustinov]

it does not work - it take v4 as a client 😞

[niketn]

@oustinov, I am surprised as to how your first code is working. Ideally you are supposed to escape backslashes in regular expression with a forward slash. Following is how your second query should look like. You should also try to test regular expressions on regex101.com

search sourcetype=apache "/apps/public/v4/client1/local/"
| rex "\/apps\/public\/v4\/(?<client>\w+)\/(?<region>\w+)\/"

Try the following run anywhere search for testing:

| makeresults
| eval _raw="/apps/public/v4/client1/local/"
| rex "\/apps\/public\/v4\/(?<client>\w+)\/(?<region>\w+)\/"

[oustinov]

Hello,
thanks for answer, but... 1st (without "/v4/") works in both variants, 2nd - same result - no fields extracted 😞

search sourcetype=apache  "/apps/public/v4/*" | rex field=_raw "/apps/public/v4/(?<client>\w+)/(?<region>\w+)/"

It finds all entries with "/apps/public/v4/" but does not extract fields at all. by delete /v4/ - it put "v4" as a client name 😞
does splunk have any restrictions on "v4"? 🙂

[niketn]

Did you check out the run anywhere search. If the run anywhere search is working but not with your raw data may be the pattern is different than what you have provided. Add some sample of raw events in the case. Mock/anonymize any sensitive data from the event keeping the pattern similar to what is present.

Does both URL v4 and non-v4 end with region? or is there any content after region. Is the URL followed by Space or Double Quotes or any other pattern?

Please add some samples.

Since the provided answer did not work as expected I have converted my answer to comment. So that other community experts can also look at this unanswered question and reply.

[FrankVl]

Are you very sure this is how you entered them? What are those " doing before rex and at the end of the line? Are you sure you actually typed those into the search bar for both attempts?

[oustinov]

sorry " before rex is a typo

[elliotproebstel]

Can you edit this and wrap the middle two lines with the code button (or single backticks, if you can't get the code button to work for you)? The formatter is eating some of your rex, which makes it harder to diagnose!