Splunk Search

How to identify IIS Application Pool status changes

jsmithn
Path Finder

I know similar questions have been asked a number of times but trying to follow the suggestions given I still cannot get it to work. Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?

I want the search to show start/stop status changes which we'll use in an actionable alert. Output items should include host, apppool, time of last change, and current status.

The Get-IISAppPool output (ingested into Splunk) looks like this:
alt text

0 Karma
1 Solution

to4kawa
Ultra Champion

recommend:

your_search
| multikv forceheader=1
| table _time host Name  Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats last(_time) as _time last(Status) as current_Status dc(Status) as Status_change by Name host
| where Status_change > 1

If all field is extracted, maybe works.


previous answer:

| makeresults 
| eval _raw="
Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
----                ------   -------  -------------  ----------
DefaultAppPool      Started  V4.0     Integrated     OnDemand
.NET V4.5 Classic   Started  V4.0     Classic        OnDemand
.NET v4.5           Started  V4.0     Integrated     OnDemand
Monkey              Started  V4.0     Integrated     OnDemand"
`comment("this is your sample")`
`comment("from here, the logic")`
| multikv forceheader=1
| table Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats dc(Status) as Status_change by Name
| where Status_change > 1

Hi, how about this?
Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?
No, you should provide sample TEXT log.

View solution in original post

0 Karma

jsmithn
Path Finder

@to4kawa, thanks for the answer. I modified it slightly just for formatting purposes and to grab the "latest" instead of "last" event. Final query follows:
my_search
| multikv forceheader=1
| table _time host Name Status
| stats latest(_time) as _time latest(Status) as Status dc(Status) as SC by host Name
| where SC > 1
| table _time host Name Status

0 Karma

alexlexxy
Explorer

@jsmithn  I have the same requirement at work, could you please post a walkthrough how you were able to get the fields such as status etc. The logs does not show these fields. Any help will be appreciated. 

0 Karma

to4kawa
Ultra Champion

I see, happy splunking 🙂

0 Karma

spayneort
Contributor

Try adding | ConvertTo-Json -Compress to your PowerShell command.

0 Karma

to4kawa
Ultra Champion

recommend:

your_search
| multikv forceheader=1
| table _time host Name  Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats last(_time) as _time last(Status) as current_Status dc(Status) as Status_change by Name host
| where Status_change > 1

If all field is extracted, maybe works.


previous answer:

| makeresults 
| eval _raw="
Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
----                ------   -------  -------------  ----------
DefaultAppPool      Started  V4.0     Integrated     OnDemand
.NET V4.5 Classic   Started  V4.0     Classic        OnDemand
.NET v4.5           Started  V4.0     Integrated     OnDemand
Monkey              Started  V4.0     Integrated     OnDemand"
`comment("this is your sample")`
`comment("from here, the logic")`
| multikv forceheader=1
| table Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats dc(Status) as Status_change by Name
| where Status_change > 1

Hi, how about this?
Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?
No, you should provide sample TEXT log.

0 Karma

jsmithn
Path Finder

Your solution does identify situations where dc(Status) > 1, so it correctly finds situations when the service state has changed over a given period. But how would I still display the host, time of last change, and current status?

I can add "host" to my display parameter stats dc(Status) AS Status_change BY host,Name but if I add the other columns the dc count becomes invalid.

0 Karma

to4kawa
Ultra Champion

how would I still display the host, time of last change, and current status?
There is no sample text log.
host? time? I can't see them.

0 Karma

jsmithn
Path Finder

Aren't host and _time native parameters splunk includes as part of every event ingested? There is no logfile; the powershell script calls the Get-IISAppPool object and the output of that is directly ingested, which is the output I originally included.

I've modified your suggestion like this but it's kludgy:

host=xyz sourcetype=IISAppPool earliest=-5m | multikv forceheader=1
| table host _time Name Status
| append
[ search host=xyz sourcetype=IISAppPool earliest=1h| multikv forceheader=1
| stats dc(Status) AS SC BY host,Name
| where SC > 1 ]
| table host _time Name Status

It shows all the AppPools on a given host when I really only want to display the status of the AppPools where SC > 1. Ideally instead of showing the impacted AppPool twice, it would just show once, and include the time of the last change and the current status.

0 Karma

to4kawa
Ultra Champion

my latest answer is checked?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...