Splunk Search

How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka
New Member

I have a splunk search which has multikv and regex.

index=os OR index=advantage sourcetype="*nmon*" |multikv|rex field=_raw "(?m)^(?P([A-Za-z])[^,]+)"| dedup type|table type| sort type

I would like to create a field in field extractions named type for regex "(?m)^(?P([A-Za-z])[^,]+)".
Regex seems to work with only multikv. Can anyone help on how to create a field which uses a regex and multikv?

My single event looks like this:
AAA,note0, Warning - use the UNIX sort command to order this file before loading into a spreadsheet
AAA,note1, The First Column is simply to get the output sorted in the right order
AAA,note2, The T0001-T9999 column is a snapshot number. To work out the actual time; see the ZZZ section at the end
CPU_ALL,CPU Total esbqpas11,User%,Sys%,Wait%,Idle%,Busy,PhysicalCPUs
PCPU_ALL,PCPU Total esbqpas11,User ,Sys ,Wait ,Idle , Entitled Capacity
SCPU_ALL,SCPU Total esbqpas11,User ,Sys ,Wait ,Idle

PCPU01,PCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
PCPU02,PCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU01,SCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
SCPU02,SCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU03,SCPU 3 esbqpas11,User ,Sys ,Wait ,Idle
SCPU04,SCPU 4 esbqpas11,User ,Sys ,Wait ,Idle
SCPU05,SCPU 5 esbqpas11,User ,Sys ,Wait ,Idle
SCPU06,SCPU 6 esbqpas11,User ,Sys ,Wait ,Idle
MEM,Memory esbqpas11,Real Free %,Virtual free %,Real free(MB),Virtual free(MB),Real total(MB),Virtual total(MB)
MEMNEW,Memory New esbqpas11,Process%,FScache%,System%,Free%,Pinned%,User%
MEMUSE,Memory Use esbqpas11,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient, lruable pages
PAGE,Paging esbqpas11,faults,pgin,pgout,pgsin,pgsout,reclaims,scans,cycles
PROC,Processes esbqpas11,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio
FILE,File I/O esbqpas11,iget,namei,dirblk,readch,writech,ttyrawch,ttycanch,ttyoutch

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

I believe this is what you want:

|rex "(?m)^(?P<type>([A-Za-z])[^,]+)" max_match=0

If you check out the doc on the rex command you'll see that max_match= Controls the number of times the regex is matched. If greater than 1, the resulting fields will be multivalued fields. Defaults to 1, use 0 to mean unlimited.

I'm not totally sure what you want to do with it after that, but you'll have a multivalue field with all the TYPE values as you designated... you can use eval and any number of mv... functions to manipulate the values.

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

d29priyanka
New Member

Thank you for your reply.Now i get all the values into type field as below
TOP
NETSIZE
DISKRIO
DISKWIO
DISKRXFER

DISKWRITE

DISKXFER

But once i get values can i search by each type?
Below query doesnt seem to work ..can someone help me with this?

For example: index=os sourcetype="nmon" |rex "(?m)^(?P([A-Za-z])[^,]+)" max_match=0|search type=NETSIZE

0 Karma

tonykung
New Member

we should ask ppl to post at least an event. 😄

0 Karma

d29priyanka
New Member

My single event looks like this:

AAA,note0, Warning - use the UNIX sort command to order this file before loading into a spreadsheet
AAA,note1, The First Column is simply to get the output sorted in the right order
AAA,note2, The T0001-T9999 column is a snapshot number. To work out the actual time; see the ZZZ section at the end
CPU_ALL,CPU Total esbqpas11,User%,Sys%,Wait%,Idle%,Busy,PhysicalCPUs
PCPU_ALL,PCPU Total esbqpas11,User ,Sys ,Wait ,Idle , Entitled Capacity
SCPU_ALL,SCPU Total esbqpas11,User ,Sys ,Wait ,Idle
PCPU01,PCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
PCPU02,PCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU01,SCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
SCPU02,SCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU03,SCPU 3 esbqpas11,User ,Sys ,Wait ,Idle
MEM,Memory esbqpas11,Real Free %,Virtual free %,Real free(MB),Virtual free(MB),Real total(MB),Virtual total(MB)
MEMNEW,Memory New esbqpas11,Process%,FScache%,System%,Free%,Pinned%,User%
MEMUSE,Memory Use esbqpas11,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient, lruable pages
PAGE,Paging esbqpas11,faults,pgin,pgout,pgsin,pgsout,reclaims,scans,cycles
PROC,Processes esbqpas11,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio
FILE,File I/O esbqpas11,iget,namei,dirblk,readch,writech,ttyrawch,ttycanch,ttyoutch

0 Karma

tonykung
New Member

your regex seems not matching any part. which character in which line you are actually looking for to be this type?

0 Karma

d29priyanka
New Member

Actually I'm looking for all the first words before the comma i.e; AAA CPU_ALL MEM etc

0 Karma

d29priyanka
New Member

Can someone help me to create a regex without using multikv?

index=os OR index=advantage sourcetype=nmon* | multikv|rex field=_raw "(?P[^,]+)"| dedup type|table type| sort type
0 Karma

tonykung
New Member

if i were you i will create fields for all like this

| rex "(?^[a-zA-z0-9]+)\s*,\s*note0" | rex "(?^[a-zA-z0-9]+)\s*,\s*note1"  ......
0 Karma

d29priyanka
New Member

But i need output as below into a field named type so that i could create a field :
AAA
CPU_ALL
PCPU_ALL
SCPU_ALL
..
..
...

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...