topic Re: Need to extract fields in Splunk Search

Need to extract fields

ashu_g50 — Mon, 28 Sep 2020 12:50:35 GMT

Hi,

I have the following in my logs

dataSetListCountInfo_HKG_generic=2
dataSetListCountInfoicm=72
dataSetListCountInfo_rm_strips=1

what I want to achieve is
HKG_generic 2
icm 72
rm_strips 1

I have tried this rex field=raw "dataSetListCountInfo\w(?.)=\w(?.)" | stats sum(datacount) by dataset

Can someone please help

Re: Need to extract fields

smolcj — Wed, 21 Nov 2012 11:08:28 GMT

i think the rex (?i)dataSetListCountInfo(?P[^=]+)=(?P[^ ]+)
will work
so update the search as
rex field=raw "(?i)dataSetListCountInfo(?P[^=]+)=(?P[^ ]+)" | top dataset,datacount|table dtatset datacount

Thank you

Re: Need to extract fields

ashu_g50 — Wed, 21 Nov 2012 11:14:12 GMT

Indeed it worked, just had to tweak it a lil

rex field=raw "(?i)dataSetListCountInfo(?P[^=]+)=(?P[^ ])" | stats sum(datacount) by dataset

Good one thanks mate!

Re: Need to extract fields

ashu_g50 — Mon, 28 Sep 2020 12:50:41 GMT

Hi Apologies,

But theres a problem with the query

It only takes the 1st instance for eg

lets say the log shows

........
dataSetListCountInfoHKG_generic=2
dataSetListCountInfoicm=72
dataSetListCountInfo__rm_strips=1
.......

.....
dataSetListCountInfoHKG_generic=6
dataSetListCountInfoicm=9
dataSetListCountInfo__rm_strips=5
........

So what I want is
HKG_generic 8
icm 81
rm_strips 6

But the query would give me

HKG_generic 8

the other two instances are ignored, how do I take into account the others ?

Sorry if I wasnt clear thanks

Re: Need to extract fields

smolcj — Thu, 22 Nov 2012 08:30:04 GMT

All these values are in same event? can u give an outline of your log?

Re: Need to extract fields

ashu_g50 — Thu, 22 Nov 2012 09:26:57 GMT

no these are different eventsmoreover its not only these fields other fileds as well like eg below

Re: Need to extract fields

ashu_g50 — Mon, 28 Sep 2020 12:51:00 GMT

11/21/12
6:36:11.302 PM 2012-11-21 18:36:11,302 INFO [accounting-service:riskserver-query-service:ManagedQuery", processID="null", analysisDefinitionID=1648795233, metainfoID=2418180724
.................
modelTypeAndNumberSeenList_bondFutureOptionModel=1modelTypeAndNumberSeenListswaptionModel=5modelTypeAndNumberSeenListgenericMortgageBackedSecurityModel=1dataSetListCountInfocorp_zero=32dataSetListCountInfoebz_europe=6dataSetListCountInfoicm=362dataSetListCountInfombs=2dataSetListCountInform_iso=2dataSetListCountInform_strips=2localPricings_localPricings=382400..............

Re: Need to extract fields

ashu_g50 — Mon, 28 Sep 2020 12:51:08 GMT

heres te thing when I use
rex field=raw "(?i)dataSetListCountInfo_(?P[^=]+)=(?Pd{1,3}+)" | stats sum(datacount) by dataset I get correct datacount numbers but wrong number of rows as the query only pics the 1st instance from different instances.dataset sum(datacount)
HKG_generic 36
ca_corp_zero 2
corp_zero 280
ebz_europe 2
icm 58

Re: Need to extract fields

ashu_g50 — Mon, 28 Sep 2020 12:51:12 GMT

now when i use
rex field=raw max_match=20 "(?i)dataSetListCountInfo_(?P[^=]+)=(?Pd{1,3}+)" | stats sum(datacount) by dataset, I get the correct number of rows expected but you see the datacount value? its wrong.dataset sum(datacount)
HKG_generic 2294
ca_corp_zero 1534
corp_zero 5185
ebz_europe 5211
icm 6471
mbs 4993
rm_agency 1594
rm_iso 4993
rm_strips 6193