topic Re: extracting fields between pattern in a search and and calculating length of value in Splunk Search

extracting fields between pattern in a search and and calculating length of value

shere — Tue, 29 Sep 2020 11:32:54 GMT

Hello.

I have a simmilar quesiton to this :
https://answers.splunk.com/answers/176585/how-to-extract-a-field-between-two-patterns-in-a-s.html

I want to do something similar for Get Request strings with patterns similar to this :

GET /~/rest/collection?ghostAccountPrompts=LP36,IL46,ID59&hyperlinkPrompts

between ? AND = is the field_Name . in the above the field_Name is ghostAccountPrompts

I need to extract everything between field_Name= and & pattern so i can do stats searches include calculate the length of the value of the field. so in the above I need to calculate the length of value LP36,IL46,ID59 which in this example is 14.

what I want to end up with is a table with Extracted field names with length and Count
Field Name Length Count
ABC 20 3
ABC 10 1
DEF 10 4

can you help ?

Re: extracting fields between pattern in a search and and calculating length of value

javiergn — Tue, 25 Oct 2016 15:08:43 GMT

Is this what you are looking for?

| rex  "\?(?<key>[^=]+)=(?<value>[^&]+)&"
| eval {key} = value
| fields - key, value

Example:

| stats count | fields - count
| eval _raw ="GET /~/rest/collection?ghostAccountPrompts=LP36,IL46,ID59&hyperlinkPrompts"
| rex  "\?(?<key>[^=]+)=(?<value>[^&]+)&"
| eval {key} = value
| fields - key, value

Output (see picture below):

Re: extracting fields between pattern in a search and and calculating length of value

richgalloway — Tue, 25 Oct 2016 15:11:38 GMT

This should get you started.

... | rex "\?(?<Field>[^=]*)=(?<fieldValue>[^&]+)" | eval Length = len(fieldValue) | stats count as Count, values(Length) as Length by Field | table Field Length Count

Re: extracting fields between pattern in a search and and calculating length of value

gokadroid — Tue, 25 Oct 2016 15:14:12 GMT

Since its a case of searching between = and & did you try this

yourSearch
| rex "\=(?<valueOfField>[^\&]+)\&.*"
| eval length=len(valueOfField)
|eval numArgs = mvcount(split(valueOfField,","))
| table valueofField, length, numArgs

Re: extracting fields between pattern in a search and and calculating length of value

shere — Tue, 25 Oct 2016 16:19:59 GMT

Thanks for the prompt reply, this would of hit the nail on the head if it wasn't for something I missed. So for a single field in the get request this will work, but there is actually two fields...
so need to now split the fields and achieve the same thing. So basically one get request string can contain one or more fields :

GET /~/rest/collection?ghostAccountPrompts=IL46&hyperlinkPrompts=IL59,IL53,IC21 HTTP/1.1\r\nHost

ghostAccountPrompts is one field - get the length of the value between Prompts= and &
hyperlinkPrompts is another field - get the length of the value between Prompts= and HTTP

Need to extract the fields and calculate length of values of the fields

Need to search string GET /~/rest/collection?
Extract the fields
calculate the length of the field values
put it in a table with each instance for field and length
if possible a count for each Field Name and number of times same length is matched

Field Length Count
ABC 20 number of time ABC is length 20
ABC 10 number of time ABC is length 10
DEF 10

Re: extracting fields between pattern in a search and and calculating length of value

richgalloway — Tue, 25 Oct 2016 18:19:17 GMT

Try this. It will do the matching, but you'll have to play with the rest a bit to get the multi-value fields to expand right. I don't have a lot of time to work on it today.

... | rex max_match=0 "(?:\?|&)(?<Field>[^=]*)=(?<fieldValue>[^& ]+)" | mvexpand Field | eval Length = len(fieldValue) | stats count as Count, values(Length) as Length by Field | table Field Length Count

Re: extracting fields between pattern in a search and and calculating length of value

shere — Mon, 31 Oct 2016 12:24:37 GMT

All, both Rich and Gokadroid provided the solution

Rich's search extracted the fields correctly and Gokadroid search calculated the value of the field correctly. I had to combine both answers to get what I wanted. I could not get the multi value fields to be extracted and calculated in one search, so I split each search by the pattern. I also used Rich's search to find all the multivalue fields , it did not correctly give me the values, but at least I identified them so I could do separate searches for them

for ghostAccountPrompts I searched between = to &:
mysearch | rex "\?(?[^=])=(?[^&]+)&."
| eval length=len(valueOfField)
|eval numArgs = mvcount(split(valueOfField,","))
| table Field, valueofField, length, numArgs

for hyperlinkPrompts I searchd between = to HTTP:

mysearch | rex "\?(?[^=])=(?[^=]+)\HTTP."
| eval length=len(valueOfField)
|eval numArgs = mvcount(split(valueOfField,","))
| table Field, length, numArgs

thanks for all your help. Secondly if someone now can see the missing piece of the puzzle to expand the multi-value fields in one search then it will be useful learning exercise for me and am keen to try it.