Getting Data In

How to report on the top 10 fields when logs have a variable number of key value pairs that are numeric?

plynch52
Explorer

I have log records with a variable number of KV (key value) pairs. Both the field and the values are numeric. The following search parses all of the log records correctly and builds a very long row of results given that there are thousands of potential key values to sum.
I have a transforms.conf & props.conf setup

[root@splunk local]# cat transforms.conf 
[get_rule_stat]
REGEX = ([0-9]+)=([0-9]+)
FORMAT = $1::$2
MV_ADD =true
CLEAN_KEYS = false

[get_rule_id]
REGEX =(?<_KEY_1>[0-9]+)=(?<_VAL_1>[0-9]+)
MV_ADD = true
CLEAN_KEYS = false
--------
[root@splunk local]# cat props.conf 
[ScoutShield]
KV_MODE = multi
REPORT-Stats = get_rule_stat
REPORT-R-ID = get_rule_id
---




source="qa_debug.log" host="splunk.localdomain" sourcetype="ScoutShield" 
| rex field=_raw "(?ms)(?=[^N]*(?:NetDefender Rule Hits Digest|N.*NetDefender Rule Hits Digest))^(?P[^\\[]+)[^\\]\\n]*\\]\\[(?P[^\\]]+)\\]\\[(?P\\d+)\\]\\[(?P\\d+)[^\\]\\n]*\\]\\[(?P[^\\]]+)" offset_field=_extracted_fields_bounds 
| rex field=stats max_match=100 "(?\d*=\d*)" 
|stats sum(*)  
| addtotals   fieldname=TotalHits 
| rename sum(*) as * 

What I would like to do is report on the top fields (as in top 10 with largest count) or to do a timechart with only those fields where the fields belong to the top 10.

It seems as though I can only use * for the field names, and there isn't a way to sub set the names.
Any suggestions would be appreciated.

0 Karma
1 Solution

somesoni2
Revered Legend

Assuming your field extraction is working fine and you're getting fields with numeric names, give this a try

your base search | table _time 1* 2* 3* 4* 5* 6* 7* 8* 9* 0*
| untable _time fieldnames fieldvalues

Once you get one row for each field using above search, your run searches like this

Top 10 fields based on count

...above search.. | top fieldnames 

Top 10 fields based on fieldvalues column

...above search.. | sort 10 -fieldvalues

Top 10 based on sum of fieldvalues

...above search.. | stats sum(fieldvalues) as fieldvalues by fieldnames | sort 10 -fieldvalues

View solution in original post

0 Karma

somesoni2
Revered Legend

Assuming your field extraction is working fine and you're getting fields with numeric names, give this a try

your base search | table _time 1* 2* 3* 4* 5* 6* 7* 8* 9* 0*
| untable _time fieldnames fieldvalues

Once you get one row for each field using above search, your run searches like this

Top 10 fields based on count

...above search.. | top fieldnames 

Top 10 fields based on fieldvalues column

...above search.. | sort 10 -fieldvalues

Top 10 based on sum of fieldvalues

...above search.. | stats sum(fieldvalues) as fieldvalues by fieldnames | sort 10 -fieldvalues
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk App Dev Quarterly Roundup: AI, Agents, and Innovation!

Another quarter, another wave of innovation. From complex integrations to pushing the limits ...

Federated Search for Dynamic Data Self Storage Is Now Generally Available on Splunk ...

 Splunk is excited to announce the General Availability of Federated Search for Dynamic Data Self Storage ...

Index This | What has many keys but can’t unlock a door?

July 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...