Splunk Search

Creating a table from a single event with multivalue field

ColinCH
Path Finder

I have an event that has disk information like: there are hosts that have more mountpoints or less mountpoints. So I need to automate this.

     /=1828716544   
     /_crit=4392484864  
     /_max=4881121280   
     /_warn=3904897024  
     /boot=106954752    
     /boot_crit=435159040   
     /boot_max=484442112    
     /boot_warn=386924544   

Now =Ii'm trying to get a table like:
Mountpoint | used | crit ....
/ | 1828716544 | 4392484864 ...

Update:

Because there were misunderstandings, here a little bit more details:

The event can no be split because there are much more details in it that i use. The event contains only information for a single host. In the example above, there are only a few "mountpoints" in this example i have the fields /, /_crit, /_max, /_warn what are automatically extracted from Splunk.(There can be a lot more of the fields for example /var/log or /opt/ and so on So it need to be dynamic).

Now i want to calculate per "mounpoint" (like, / and /boot in the example) the usage, max...)

I was thinking it was helpful to create a MV with regex that i have all paths in a field (Path has the values / and /boot).

0 Karma
1 Solution

ColinCH
Path Finder
index=* sourcetype=mysourcetype check_command=disk hostname=myhostname
| dedup hostname 
| rex mode=sed "s/(\"\_(\/|\/[a-zA-Z\/]+))\"/\1_used\"/g"
| rex max_match=0 "(?<bbb>\"\_(\/|\/[a-zA-Z\/]+)\_\w+\"\:\d+\.\d+\,)" 
| table bbb | eval aaa=mvsort(bbb)
| rex max_match=0 field=aaa "\_(?<path>(\/|\/[a-zA-Z\/]+)\_used)\"\:(?<used>\d+)"
| rex max_match=0 field=aaa "_crit\":(?<crit>\d+)" 
| rex max_match=0 field=aaa "_max\":(?<max>\d+)"
| rex max_match=0 field=aaa "_warn\":(?<warn>\d+)" 
| eval zip=mvzip(path,used,"##") 
| eval zip=mvzip(zip,crit,"##")
| eval zip=mvzip(zip,max,"##")
| eval zip=mvzip(zip,warn,"##")
| mvexpand zip 
|rex max_match=0 field=zip "(?<path>.*)##(?<used>.*)##(?<crit>.*)##(?<max>.*)##(?<warn>.*)" 
| fields - zip 
| eval used=used/1024/1024
| eval crit=crit/1024/1024 
| eval max=max/1024/1024 
| eval warn=warn/1024/1024
| eval free=max-used
| eval percent=(used/max*100)
| eval percent=round(percent,0)
| table path free used  max percent
| rex mode=sed field=path "s/_used//" 

here is my result, the main event was JSON and the mountpoint were not sorted, so i need to modify first with sed the fields before i can sort i with mvsort correctly. Then i created some more MV-Fields with rex and create an "array" with mvzip.

For me, this is a good solution. I can process several fields per event in this way. it performs very well for me.

View solution in original post

0 Karma

ColinCH
Path Finder
index=* sourcetype=mysourcetype check_command=disk hostname=myhostname
| dedup hostname 
| rex mode=sed "s/(\"\_(\/|\/[a-zA-Z\/]+))\"/\1_used\"/g"
| rex max_match=0 "(?<bbb>\"\_(\/|\/[a-zA-Z\/]+)\_\w+\"\:\d+\.\d+\,)" 
| table bbb | eval aaa=mvsort(bbb)
| rex max_match=0 field=aaa "\_(?<path>(\/|\/[a-zA-Z\/]+)\_used)\"\:(?<used>\d+)"
| rex max_match=0 field=aaa "_crit\":(?<crit>\d+)" 
| rex max_match=0 field=aaa "_max\":(?<max>\d+)"
| rex max_match=0 field=aaa "_warn\":(?<warn>\d+)" 
| eval zip=mvzip(path,used,"##") 
| eval zip=mvzip(zip,crit,"##")
| eval zip=mvzip(zip,max,"##")
| eval zip=mvzip(zip,warn,"##")
| mvexpand zip 
|rex max_match=0 field=zip "(?<path>.*)##(?<used>.*)##(?<crit>.*)##(?<max>.*)##(?<warn>.*)" 
| fields - zip 
| eval used=used/1024/1024
| eval crit=crit/1024/1024 
| eval max=max/1024/1024 
| eval warn=warn/1024/1024
| eval free=max-used
| eval percent=(used/max*100)
| eval percent=round(percent,0)
| table path free used  max percent
| rex mode=sed field=path "s/_used//" 

here is my result, the main event was JSON and the mountpoint were not sorted, so i need to modify first with sed the fields before i can sort i with mvsort correctly. Then i created some more MV-Fields with rex and create an "array" with mvzip.

For me, this is a good solution. I can process several fields per event in this way. it performs very well for me.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

If you are trying to get a horizontal table of all mountpoints for each host, then this will calculate them.

 your base search which includes hostname and multivalued field Mountpoint 
 | table hostname Mountpoint
 | mvexpand Mountpoint 
 | rex field=Mountpoint "(?<Mountpoint>[^\=]+)\=(?<used>\d+)" 

 | rename COMMENT as "Deal with things that wouldn't be valid variable names"
 | replace "\\" WITH "XX_" IN Mountpoint
 | eval {Mountpoint} = used
 | fields - Mountpoint used
 | stats values(*) as * by hostname

At this point, the values are splayed out horizontally, but I'm not sure how useful that will be to look at. Using chart with useother=f will give you a pretty wide thing to look at.

Unless you have a reason to want to compare the mountpoints across hosts, you might be better off going this way...

 your base search which includes hostname and multivalued field Mountpoint 
 | table hostname Mountpoint
 | mvexpand Mountpoint 
 | rex field=Mountpoint "(?<Mountpoint>[^\=]+)\=(?<used>\d+)" 
 | sort 0 hostname Mountpoint
 | stats list(Mountpoint) as Mountpoint  list(used) as used by hostname
0 Karma

ColinCH
Path Finder

hi, thanks for your answer. I don't think that works. I update the question, that the question is more understandable.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

@ColinCH - I don't recognize why _crit would roll up, or what the rest of your desired chart might look like.

@somesoni2's answer will work if you are willing to take each individual mountpount as listed.

If you want to roll up the mountpoints, like if everything under / needs to be tabled somehow as belonging to /, and everything under /foo/bar needs to be rolled up to /foo, then you should give us a clear picture of what the specs might be, with example data and the layout, more than a single line.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try like this

your base search which includes multivalued field Mountpoint 
| mvexpand Mountpoint 
| rex field=Mountpoint "(?<Mountpoint>[^\=]+)\=(?<used>\d+)" 
| table Mountpoint used ..anyother field you need

ColinCH
Path Finder

Thanks, but that is not what i want. I've updates my initial question. So there are much more detail now.

0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

Are the values consistent in the multivalue field? If so, you could use mvindex to create a field for each.

YOUR BASE SEARCH | | eval field1=mvindex(bytes,0) | eval field2=mvindex(bytes,1)
0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...