topic Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search? in Splunk Search

How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka — Mon, 28 Sep 2020 19:22:25 GMT

I have a splunk search which has multikv and regex.

index=os OR index=advantage sourcetype="*nmon*" |multikv|rex field=_raw "(?m)^(?P([A-Za-z])[^,]+)"| dedup type|table type| sort type

I would like to create a field in field extractions named type for regex "(?m)^(?P([A-Za-z])[^,]+)".
Regex seems to work with only multikv. Can anyone help on how to create a field which uses a regex and multikv?

My single event looks like this:
AAA,note0, Warning - use the UNIX sort command to order this file before loading into a spreadsheet
AAA,note1, The First Column is simply to get the output sorted in the right order
AAA,note2, The T0001-T9999 column is a snapshot number. To work out the actual time; see the ZZZ section at the end
CPU_ALL,CPU Total esbqpas11,User%,Sys%,Wait%,Idle%,Busy,PhysicalCPUs
PCPU_ALL,PCPU Total esbqpas11,User ,Sys ,Wait ,Idle , Entitled Capacity
SCPU_ALL,SCPU Total esbqpas11,User ,Sys ,Wait ,Idle

PCPU01,PCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
PCPU02,PCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU01,SCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
SCPU02,SCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU03,SCPU 3 esbqpas11,User ,Sys ,Wait ,Idle
SCPU04,SCPU 4 esbqpas11,User ,Sys ,Wait ,Idle
SCPU05,SCPU 5 esbqpas11,User ,Sys ,Wait ,Idle
SCPU06,SCPU 6 esbqpas11,User ,Sys ,Wait ,Idle
MEM,Memory esbqpas11,Real Free %,Virtual free %,Real free(MB),Virtual free(MB),Real total(MB),Virtual total(MB)
MEMNEW,Memory New esbqpas11,Process%,FScache%,System%,Free%,Pinned%,User%
MEMUSE,Memory Use esbqpas11,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient, lruable pages
PAGE,Paging esbqpas11,faults,pgin,pgout,pgsin,pgsout,reclaims,scans,cycles
PROC,Processes esbqpas11,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio
FILE,File I/O esbqpas11,iget,namei,dirblk,readch,writech,ttyrawch,ttycanch,ttyoutch

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

tonykung — Wed, 01 Apr 2015 17:30:33 GMT

we should ask ppl to post at least an event. 😄

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka — Mon, 28 Sep 2020 19:22:28 GMT

My single event looks like this:

AAA,note0, Warning - use the UNIX sort command to order this file before loading into a spreadsheet
AAA,note1, The First Column is simply to get the output sorted in the right order
AAA,note2, The T0001-T9999 column is a snapshot number. To work out the actual time; see the ZZZ section at the end
CPU_ALL,CPU Total esbqpas11,User%,Sys%,Wait%,Idle%,Busy,PhysicalCPUs
PCPU_ALL,PCPU Total esbqpas11,User ,Sys ,Wait ,Idle , Entitled Capacity
SCPU_ALL,SCPU Total esbqpas11,User ,Sys ,Wait ,Idle
PCPU01,PCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
PCPU02,PCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU01,SCPU 1 esbqpas11,User ,Sys ,Wait ,Idle
SCPU02,SCPU 2 esbqpas11,User ,Sys ,Wait ,Idle
SCPU03,SCPU 3 esbqpas11,User ,Sys ,Wait ,Idle
MEM,Memory esbqpas11,Real Free %,Virtual free %,Real free(MB),Virtual free(MB),Real total(MB),Virtual total(MB)
MEMNEW,Memory New esbqpas11,Process%,FScache%,System%,Free%,Pinned%,User%
MEMUSE,Memory Use esbqpas11,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient, lruable pages
PAGE,Paging esbqpas11,faults,pgin,pgout,pgsin,pgsout,reclaims,scans,cycles
PROC,Processes esbqpas11,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio
FILE,File I/O esbqpas11,iget,namei,dirblk,readch,writech,ttyrawch,ttycanch,ttyoutch

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

tonykung — Wed, 01 Apr 2015 17:48:46 GMT

your regex seems not matching any part. which character in which line you are actually looking for to be this type?

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka — Wed, 01 Apr 2015 17:55:24 GMT

Actually I'm looking for all the first words before the comma i.e; AAA CPU_ALL MEM etc

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka — Wed, 01 Apr 2015 18:14:59 GMT

Can someone help me to create a regex without using multikv?

index=os OR index=advantage sourcetype=nmon* | multikv|rex field=_raw "(?P[^,]+)"| dedup type|table type| sort type

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

tonykung — Wed, 01 Apr 2015 18:28:16 GMT

if i were you i will create fields for all like this

| rex "(?^[a-zA-z0-9]+)\s*,\s*note0" | rex "(?^[a-zA-z0-9]+)\s*,\s*note1"  ......

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka — Mon, 28 Sep 2020 19:22:36 GMT

But i need output as below into a field named type so that i could create a field :
AAA
CPU_ALL
PCPU_ALL
SCPU_ALL
..
..
...

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

rsennett_splunk — Tue, 07 Apr 2015 01:43:38 GMT

I believe this is what you want:

|rex "(?m)^(?P<type>([A-Za-z])[^,]+)" max_match=0

If you check out the doc on the rex command you'll see that max_match= Controls the number of times the regex is matched. If greater than 1, the resulting fields will be multivalued fields. Defaults to 1, use 0 to mean unlimited.

I'm not totally sure what you want to do with it after that, but you'll have a multivalue field with all the TYPE values as you designated... you can use eval and any number of mv... functions to manipulate the values.

Re: How to extract and create a field named "type" using regex "(?m)^(?P([A-Za-z])[^,]+)" and multikv in a search?

d29priyanka — Tue, 07 Apr 2015 16:31:32 GMT

Thank you for your reply.Now i get all the values into type field as below
TOP
NETSIZE
DISKRIO
DISKWIO
DISKRXFER

DISKWRITE

DISKXFER

But once i get values can i search by each type?
Below query doesnt seem to work ..can someone help me with this?

For example: index=os sourcetype="nmon" |rex "(?m)^(?P([A-Za-z])[^,]+)" max_match=0|search type=NETSIZE