Splunk Search

extracting fields between pattern in a search and and calculating length of value

shere
New Member

Hello.

I have a simmilar quesiton to this :
https://answers.splunk.com/answers/176585/how-to-extract-a-field-between-two-patterns-in-a-s.html

I want to do something similar for Get Request strings with patterns similar to this :

GET /~/rest/collection?ghostAccountPrompts=LP36,IL46,ID59&hyperlinkPrompts

between ? AND = is the field_Name . in the above the field_Name is ghostAccountPrompts

I need to extract everything between field_Name= and & pattern so i can do stats searches include calculate the length of the value of the field. so in the above I need to calculate the length of value LP36,IL46,ID59 which in this example is 14.

what I want to end up with is a table with Extracted field names with length and Count
Field Name Length Count
ABC 20 3
ABC 10 1
DEF 10 4

can you help ?

Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

This should get you started.

... | rex "\?(?<Field>[^=]*)=(?<fieldValue>[^&]+)" | eval Length = len(fieldValue) | stats count as Count, values(Length) as Length by Field | table Field Length Count
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

shere
New Member

All, both Rich and Gokadroid provided the solution

Rich's search extracted the fields correctly and Gokadroid search calculated the value of the field correctly. I had to combine both answers to get what I wanted. I could not get the multi value fields to be extracted and calculated in one search, so I split each search by the pattern. I also used Rich's search to find all the multivalue fields , it did not correctly give me the values, but at least I identified them so I could do separate searches for them

for ghostAccountPrompts I searched between = to &:
mysearch | rex "\?(?[^=])=(?[^&]+)&."
| eval length=len(valueOfField)
|eval numArgs = mvcount(split(valueOfField,","))
| table Field, valueofField, length, numArgs

for hyperlinkPrompts I searchd between = to HTTP:

mysearch | rex "\?(?[^=])=(?[^=]+)\HTTP."
| eval length=len(valueOfField)
|eval numArgs = mvcount(split(valueOfField,","))
| table Field, length, numArgs

thanks for all your help. Secondly if someone now can see the missing piece of the puzzle to expand the multi-value fields in one search then it will be useful learning exercise for me and am keen to try it.

0 Karma

gokadroid
Motivator

Since its a case of searching between = and & did you try this

yourSearch
| rex "\=(?<valueOfField>[^\&]+)\&.*"
| eval length=len(valueOfField)
|eval numArgs = mvcount(split(valueOfField,","))
| table valueofField, length, numArgs
0 Karma

richgalloway
SplunkTrust
SplunkTrust

This should get you started.

... | rex "\?(?<Field>[^=]*)=(?<fieldValue>[^&]+)" | eval Length = len(fieldValue) | stats count as Count, values(Length) as Length by Field | table Field Length Count
---
If this reply helps you, Karma would be appreciated.
0 Karma

shere
New Member

Thanks for the prompt reply, this would of hit the nail on the head if it wasn't for something I missed. So for a single field in the get request this will work, but there is actually two fields...
so need to now split the fields and achieve the same thing. So basically one get request string can contain one or more fields :

GET /~/rest/collection?ghostAccountPrompts=IL46&hyperlinkPrompts=IL59,IL53,IC21 HTTP/1.1\r\nHost

ghostAccountPrompts is one field - get the length of the value between Prompts= and &
hyperlinkPrompts is another field - get the length of the value between Prompts= and HTTP

Need to extract the fields and calculate length of values of the fields

  1. Need to search string GET /~/rest/collection?
  2. Extract the fields
  3. calculate the length of the field values
  4. put it in a table with each instance for field and length
  5. if possible a count for each Field Name and number of times same length is matched

Field Length Count
ABC 20 number of time ABC is length 20
ABC 10 number of time ABC is length 10
DEF 10

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this. It will do the matching, but you'll have to play with the rest a bit to get the multi-value fields to expand right. I don't have a lot of time to work on it today.

... | rex max_match=0 "(?:\?|&)(?<Field>[^=]*)=(?<fieldValue>[^& ]+)" | mvexpand Field | eval Length = len(fieldValue) | stats count as Count, values(Length) as Length by Field | table Field Length Count
---
If this reply helps you, Karma would be appreciated.
0 Karma

javiergn
SplunkTrust
SplunkTrust

Is this what you are looking for?

| rex  "\?(?<key>[^=]+)=(?<value>[^&]+)&"
| eval {key} = value
| fields - key, value

Example:

| stats count | fields - count
| eval _raw ="GET /~/rest/collection?ghostAccountPrompts=LP36,IL46,ID59&hyperlinkPrompts"
| rex  "\?(?<key>[^=]+)=(?<value>[^&]+)&"
| eval {key} = value
| fields - key, value

Output (see picture below):

alt text

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...