How do you display the ulimit values for a group of servers in a chart so that they can be compared?

[rholm01]

index=_internal host=* source=*splunkd.log ulimit is what I found that works. I would to make sure that certain groups of servers have the same ulimit settings for consistency and to meet or exceed Splunk recommendations.

[valiquet · Accepted solution]

|rest /services/server/sysinfo  | stats  values(ulimit) by splunk_server 

Straing from Splunk web UI in 2 seconds

[FrankVl]

You'd first have to write a few rex commands to extract the ulimit values from those events, for example:

| rex field=event_message "cpu time: (?<cpu_time>\S+)"

And then the same for the other limits.

then add something like:

| stats latest(cpu_time) latest(open_files) by host

adding such a latest() for each extracted limit field.

[rholm01]

I did not get any results for the following:
| rex field=event_message "cpu time: (?\S+)" | stats latest(cpu_time) by host

Running the following command I get the ulimits value for one of my hosts (blanked out the host name:
index=_internal source=*splunkd.log ulimit host="xxxxxxxxxxxx.com"

event_message
Linux vm.overcommit setting, value="0"
Linux transparent hugepage support, enabled="never" defrag="never"
Limit: cpu time: unlimited
Limit: user processes: 16000 processes
Limit: open files: 64000 files [hard maximum: 1048576 files]
Limit: data file size: unlimited
Core file generation disabled.
Limit: core file size: 0 bytes [hard maximum: unlimited]
Limit: stack size: 8388608 bytes [hard maximum: unlimited]
Limit: resident memory size: unlimited
Limit: data segment size: unlimited
Limit: virtual address space size: unlimited

[FrankVl]

So index=_internal source=*splunkd.log ulimit host="xxxxxxxxxxxx.com" gives the result as you are showing, with that data in the event_message field?

Then I don't follow why the below wouldn't work.

index=_internal source=*splunkd.log ulimit host="xxxxxxxxxxxx.com"
| rex field=event_message "cpu time: (?<cpu_time>\S+)"
| stats latest(cpu_time) by host