topic Re: Extract value until specified char & in Getting Data In

Extract value until specified char &

saifullakhalid — Mon, 20 Nov 2017 20:11:59 GMT

I want to extract value until the first occurrence of char &

My log :

?pyActivity=FinishAssig&pzPrimaryPageName=pyWorkPage&pzTransactionId=xxxxx&pzFromFrame=pyWorkPage HTTP/1.1" 200 1383

?pyActivity=Sh-Harness&Purpose=Perform&SkipFrame=true&TaskIndex=1&HarnessMode=ACTION&TaskValue=CAIXXXr_____CA&TaskHTML=CAXXX&ReadOnly=-1&FrameName=pyWorkPage&pzPrimaryPageName=pyWorkPage HTTP/1.1" 200 14547

?pyActivity=Data-Por.Get&IsBMLogin=true&inStandardsMode=false&AJAXTrackID=1&pzHarnessID=HIDXXXXHTTP/1.1" 200 10

expected output:

FinishAssig
Sh-Harness
Data-Por.Get

I tried below but it dint work, it will extract values until last occurance of char &
rex field=_raw "\?pyActivity=(?.+)&"

Re: Extract value until specified char &

elliotproebstel — Mon, 20 Nov 2017 20:15:21 GMT

Try this:

rex field=_raw "\?pyActivity=([^&]+)&"

Re: Extract value until specified char &

MuS — Mon, 20 Nov 2017 20:17:24 GMT

can you please modify your post and use the code function (mark the text and press either CTRL-K or the 101010 icon) this helps in keeping the log and the regex 😉

Re: Extract value until specified char &

saifullakhalid — Mon, 20 Nov 2017 20:40:16 GMT

I tried this it worked, but if you have other options, please suggest.

rex field=_raw "\?pyActivity=(?\w+)&"

Re: Extract value until specified char &

elliotproebstel — Mon, 20 Nov 2017 21:10:59 GMT

Did the regex I suggested below work for you? If yours works better, list it as an answer and accept it! 🙂

Re: Extract value until specified char &

niketn — Tue, 29 Sep 2020 16:54:29 GMT

Ideally Splunk should extract key value pair on its own from your _raw data provided KV_MODE=none has not been set explicitly in props.conf. Have you run the search in SMART or VERBOSE mode to check Interesting Fileds list to confirm whether Search Time Field Discovery is working as expected or not?

During search time you can also do this via KV command or extract command by passing yourkvdelimandpairdelim(besides therexcommand which is already shared). Following are two run anywhere searches based on your data (PS: Commands till| rename` are used to mock the sample data provided in the question. You can add the KV or extract command directly after your base search filter :

|  makeresults
|  eval data="?pyActivity=FinishAssig&pzPrimaryPageName=pyWorkPage&pzTransactionId=xxxxx&pzFromFrame=pyWorkPage HTTP/1.1\" 200 1383;
?pyActivity=Sh-Harness&Purpose=Perform&SkipFrame=true&TaskIndex=1&HarnessMode=ACTION&TaskValue=CAIXXXr_____CA&TaskHTML=CAXXXℜadOnly=-1&FrameName=pyWorkPage&pzPrimaryPageName=pyWorkPage HTTP/1.1\" 200 14547;
?pyActivity=Data-Por.Get&IsBMLogin=true&StandardsMode=false&AJAXTrackID=1&pzHarnessID=HIDXXXXHTTP/1.1\" 200 10"
|  makemv data delim=";"
|  mvexpand data
|  rename data as _raw
|  KV
|  table * _raw

OR the following:

|  makeresults
|  eval data="?pyActivity=FinishAssig&pzPrimaryPageName=pyWorkPage&pzTransactionId=xxxxx&pzFromFrame=pyWorkPage HTTP/1.1\" 200 1383;
?pyActivity=Sh-Harness&Purpose=Perform&SkipFrame=true&TaskIndex=1&HarnessMode=ACTION&TaskValue=CAIXXXr_____CA&TaskHTML=CAXXXℜadOnly=-1&FrameName=pyWorkPage&pzPrimaryPageName=pyWorkPage HTTP/1.1\" 200 14547;
?pyActivity=Data-Por.Get&IsBMLogin=true&StandardsMode=false&AJAXTrackID=1&pzHarnessID=HIDXXXXHTTP/1.1\" 200 10"
|  makemv data delim=";"
|  mvexpand data
|  rename data as _raw
|  extract kvdelim="=" pairdelim="&amp;"
|  table * _raw

Re: Extract value until specified char &

saifullakhalid — Tue, 21 Nov 2017 15:19:28 GMT

No it dint work I got this error
The search job has failed due to an error. You may be able view the job in the Job Inspector.
index="clean" | rex field=_raw "\?pyActivity=([^&]+)&"

Re: Extract value until specified char &

saifullakhalid — Tue, 21 Nov 2017 15:26:54 GMT

It did not worked here is the query I am running on search

Error:
The search job has failed due to an error. You may be able view the job in the Job Inspector.

Re: Extract value until specified char &

saifullakhalid — Tue, 29 Sep 2020 16:54:59 GMT

This works for me

index="prod_clean" | rex field=_raw "\?pyActivity=(?([^&]+))" | stats count AS "Count" by PA | table "PA" "Count"

Re: Extract value until specified char &

niketn — Wed, 22 Nov 2017 01:30:44 GMT

As stated commands till rename were just to mock sample data for testing. After your base i.e. index=clean next command should be extract.

index="clean"
| extract kvdelim="=" pairdelim="&"
| table * _raw

PS: Ideally you should add specific sourcetype="<yourSourceTypeNameGoesHere>" to your base search in case your index clean is storing multiple sourcetypes. For optimized query base search should have as many filters as possible.

Re: Extract value until specified char &

mtulett_splunk — Wed, 22 Nov 2017 02:44:11 GMT

Try using: rex field=_raw "pyActivity=(?<pyActivity>.+?)&"

This will match as few characters as possible up to the first ampersand. The question mark after the plus sign is what makes this happen. You also need to name the field in the extraction, which is the ?<pyActivity> portion of the regex.

Re: Extract value until specified char &

niketn — Fri, 24 Nov 2017 04:14:21 GMT

@saifullakhalid, can you please confim whether it worked or not?