All Apps and Add-ons

Splunk for CA Siteminder: How to extract fields Current, Max, Limit, and Exceeded limit to create a custom search to retrieve load information

Hello everyone!

I'm new at using Splunk. My team installed it recently along with Splunk for CA SiteMinder (https://splunkbase.splunk.com/app/842/ ). It works fine with the default searches, but we're having a hard time writing a custom search to retrieve load information.

We want to extract the information of the fields Current, Max, Limit, and Exceeded limit to create a time line chart, this information is displayed as follow in the logs:

===================================================================================
[11627/3721370512][Sun Aug 09 2015 21:00:02][CServer.cpp:4403][INFO] System Statistics
[11627/3721370512][Sun Aug 09 2015 21:00:02][CServer.cpp:4409][INFO] Available file descriptors: 63488
[11627/3721370512][Sun Aug 09 2015 21:00:02][CServer.cpp:4420][INFO] Thread pool limit: 20
[11627/3721370512][Sun Aug 09 2015 21:00:02][CServer.cpp:4440][INFO] Thread pool: Msgs=116042596 Waits=109838215 Misses=9595616 Max HP Msg= 424 Max NP Msg=  388 Current Depth=     0 Max Depth=    812 Current High Depth=       0 Current Norm Depth=        0 Current Threads=        20 Max Threads=         20
[11627/3721370512][Sun Aug 09 2015 21:00:02][CServer.cpp:4448][INFO] Connections: Current=289 Max=1349 Limit=10000 Exceeded limit=   0
[11627/3721370512][Sun Aug 09 2015 21:00:02][CServer.cpp:4451][INFO] ===================================================================================

Any help is greatly appreciated.

Cheers

1 Solution

SplunkTrust
SplunkTrust

If you do not have those fields, you can create them with something like the following appended to your search:

... | rex "Current=(?<current>\d+)\s+Max=(?<max>\d+)\s+Limit=(?<limit>\d+)\s+Exceeded limit=(?<exceeded_limit>\d+)

Once you have the fields showing up (either because they're there already or because you told Splunk how to create them with the rex command above), a timechart with all values could be added by appending a bit more:

... | timechart current, max, limit, exceeded_limit 

It looks like you may have a scale issue with Exceeded_limit, since it's likely low compared to the rest. You could, once you've built the timechart, use that field as an overlay using the second Y axis.

View solution in original post

SplunkTrust
SplunkTrust

If you do not have those fields, you can create them with something like the following appended to your search:

... | rex "Current=(?<current>\d+)\s+Max=(?<max>\d+)\s+Limit=(?<limit>\d+)\s+Exceeded limit=(?<exceeded_limit>\d+)

Once you have the fields showing up (either because they're there already or because you told Splunk how to create them with the rex command above), a timechart with all values could be added by appending a bit more:

... | timechart current, max, limit, exceeded_limit 

It looks like you may have a scale issue with Exceeded_limit, since it's likely low compared to the rest. You could, once you've built the timechart, use that field as an overlay using the second Y axis.

View solution in original post

I created the new fields using the Extract New Fields (http://docs.splunk.com/Documentation/Splunk/6.2.5/Knowledge/FXSelectFieldsstep) functionality and after that I was able to generate the chart with the search "... | timechart max(Current) max(exceeded) by host".

Thanks for your help

Cheers

0 Karma

Path Finder

Hi tiago_comasseto,
we are also planning to use splunk for monitoring siteminder.
Can you give me some brief overview, How you are configuring the dashboards for this?
what type of alerts have you configured and how are you planning to implement the performace calculations for this.
Please help understanding this APP.

0 Karma

Hi, I'm sorry to say that we've abandoned Splunk, We now use Graylog, my colleagues and I developed a content pack for Siteminder that can be downloaded here: https://marketplace.graylog.org/addons/59470454-c173-4a17-ae23-f6f90c2bb234

I just want to mention that Splunk is much better than Graylog, but also much expensive, that's the reason we moved to another product.

Cheers

0 Karma

SplunkTrust
SplunkTrust

I'm surprised they're not already extracted - Splunk does a pretty good job with anything that's obviously a key-value pair, like "Current=289".

So, if you perform a search in Splunk that returns the above lines, do appropriate fields appear on the left? You may have to change your Discovery Mode (way on the right under the time selector) to Verbose Mode.

Let us know about that because whether those are there or not will impact the answer significantly.

0 Karma

Hi there, thanks for your comment, the fields do appear in the left side, the problem is that I couldn't find a way to display a time chart with the value of the fields, here's a screenshot of what I'm seeing: http://awesomescreenshot.com/0f456g00a0

0 Karma

SplunkTrust
SplunkTrust

Well, now, THAT's interesting!

My guess is a bad regex in transforms.conf or props.conf. More on that in a bit, but first:

A workaround (and not foolhardy, either) would be to do the rex I mention below to get new "current" and other values (notice all lowercase). If those new extractions work, try the timechart as suggested below too. That'll at least get you working for now and until we can fix this "better."

Then, we'll continue figuring out why they're broken out of the box. Again, I suspect $splunkhome$/etc//default/props.conf and transforms.conf are incorrect. If you search those for your eventtype (I can't quite read it from the screenshot), you should find stanzas with REGEX lines. Those are your culprits, I think.

0 Karma