topic Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2 in Splunk Search

How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Tue, 29 Sep 2020 10:41:55 GMT

Hi,

I have the following search:

host="*beta*" index=wls OR index=main sourcetype=wls_managedserver OR source="/etc/httpd/logs/portal-access_log*" |rex field=message_text "UCE-(?< UCE_Code >[^\s\:;]+)"|rex "UCE-(?< UCE_Code1 >[^\"]+)"|table UCE_Code UCE_Code1

The search works if I break it apart, meaning using:

Index=wls and sourcetype=wls_managedserver with rex UCE-(?< UCE_Code >[^\s\:;]+)" as query1

index=main source="/etc/httpd/logs/portal-access_log*" with rex "UCE-(?< UCE_Code1 >[^\"]+)" as query2
but of course I'd like to combine the two searches. How can I have the rex definitions associate to a sourcetype/index/source?

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

sundareshr — Thu, 18 Aug 2016 21:45:57 GMT

How about something like this.

host="*beta*" index=wls OR index=main sourcetype=wls_managedserver OR source="/etc/httpd/logs/portal-access_log*" | rex field=message_text "UCE-(?< UCE_Code >[^\s\:;]+)"| rex "UCE-(?< UCE_Code1 >[^\"]+)" | eval UCE_Code=if(source="/etc/httpd/logs/portal-access_log*", UCE_CODE1, UCE_CODE) | table UCE_Code

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Thu, 18 Aug 2016 21:55:08 GMT

Hi sundareshr, thats close! One problem though, the rex extraction for UCE_code1 is pulling the wrong data because it is looking at a different log than intended.

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

sundareshr — Thu, 18 Aug 2016 21:59:07 GMT

can you share a sample for each type (source="/etc/httpd/logs/portal-access_log*" AND the other)

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Tue, 29 Sep 2020 10:41:58 GMT

a bit more detail.....

rex field=message_text "UCE-(?< UCE_Code >[^\s:;]+)" gets the right UCE Code if it is looking at index=wls sourcetype=wls_managedserver events

similarly rex "UCE-(?< UCE_Code1 >[^\"]+)" gets the right UCE Code if it is looking at index=main source="/etc/httpd/logs/portal-access*log*" events

but when the sourcetype/index/sources are cobbled together the rex gets confused because the formats don't match if you switch them.

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

sundareshr — Thu, 18 Aug 2016 22:03:10 GMT

I understand. I am wondering if there is a different rex that could be applied to make this work without two searches. Can you share couple of samples, you can obfuscate any sensitive info.

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Tue, 29 Sep 2020 10:42:01 GMT

rex field=message_text "UCE-(?< UCE_Code >[^\s:;]+)" index=wls sourcetype=wls_managedserver events

< Aug 18, 2016 5:05:20 PM CDT > < Warning > < ucontrol > < betamax-cpe2 > < managedServer2 > < pool-5-thread-4 > << anonymous >> < > < > < 1471557920294 > < BEA-000000 > < fn.xmpp.v2.IQSMAPHandler - UCE-24100 - Server exception while processing SMAP message '/event/lighting' from '1000275@xmpp/9c972687947f'

rex "UCE-(?< UCE_Code1 >[^\"]+)" index=main source="/etc/httpd/logs/portal-access*log*" events

"beta.icontrol.com" 99.98.192.121 "99.98.192.121" - - [18/Aug/2016:17:07:22 -0500] "GET /rest/icontrol/sites/1001226/network/instances/1004a1514a4f98.0/functions/getUserCodes HTTP/1.1" 404 43 0 10418 "-" "Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-G930A Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.98 Mobile Safari/537.36" "UCE-16001"

"beta.icontrol.com" 54.174.106.18 "54.174.106.18" - - [18/Aug/2016:17:13:28 -0500] "GET /rest/icontrol/users/1000251/preferences/scenes HTTP/1.1" 404 46 0 22809 "-" "Apache-HttpClient/4.3.6 (java 1.5)" "UCE-16000"

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

sundareshr — Thu, 18 Aug 2016 22:35:54 GMT

Try this regex `"UCE-(?(\d+\")|(\d+\s.*))"

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

sundareshr — Thu, 18 Aug 2016 22:57:26 GMT

Try this run-anywhere sample. (use everything after the | table x segment)

| gentimes start=-1 | eval x="< Warning > < ucontrol > < betamax-cpe2 > < managedServer2 > < pool-5-thread-4 > << anonymous >> < > < > < 1471557920294 > < BEA-000000 > < fn.xmpp.v2.IQSMAPHandler - UCE-24100 - Server exception while processing SMAP message '/event/lighting' from '1000275@xmpp/9c972687947f'|

 - - [18/Aug/2016:17:07:22 -0500] \"GET /rest/icontrol/sites/1001226/network/instances/1004a1514a4f98.0/functions/getUserCodes HTTP/1.1\" 404 43 0 10418 \"-\" \"Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-G930A Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.98 Mobile Safari/537.36\" \"UCE-16001\"|

 - - [18/Aug/2016:17:13:28 -0500] \"GET /rest/icontrol/users/1000251/preferences/scenes HTTP/1.1\" 404 46 0 22809 \"-\" \"Apache-HttpClient/4.3.6 (java 1.5)\" \"UCE-16000\""| makemv x delim="|" | mvexpand x | table x | rex field=x "UCE-(?<code>(\d+\")|(\d+\s.*))" | table x code | rex mode=sed field=code "s/(\d+)\"/\1/g"

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Fri, 19 Aug 2016 14:32:16 GMT

Hi Sundareshr,

I think you are very close. The table does have the UCE code (numeric digits) but it also has the error description which is pretty technical and won't be understood by the users. I've made a lookup table that replaces the description with a more user friendly one. How can the rex be modified so the UCE code is only the digits?

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Fri, 19 Aug 2016 14:41:03 GMT

Think I figured it out UCE-(?< code >(\d+))

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

sundareshr — Fri, 19 Aug 2016 14:49:12 GMT

If all you need is the numbers, you just need this. You don't need the sed

... | rex field=x "UCE-(?<code>\d+) | ...

Re: How to create a search with conditional field extraction logic? If sourcetype=x, use rex extraction 1, but if source=y, use rex extraction 2

dbcase — Fri, 19 Aug 2016 14:59:24 GMT

Thanks (again) Sundareshr!!