topic How to identify IIS Application Pool status changes in Splunk Search

How to identify IIS Application Pool status changes

jsmithn — Fri, 31 Jan 2020 02:08:34 GMT

I know similar questions have been asked a number of times but trying to follow the suggestions given I still cannot get it to work. Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?

I want the search to show start/stop status changes which we'll use in an actionable alert. Output items should include host, apppool, time of last change, and current status.

The Get-IISAppPool output (ingested into Splunk) looks like this:

Re: How to identify IIS Application Pool status changes

to4kawa — Sat, 01 Feb 2020 04:10:43 GMT

recommend:

your_search
| multikv forceheader=1
| table _time host Name  Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats last(_time) as _time last(Status) as current_Status dc(Status) as Status_change by Name host
| where Status_change > 1

If all field is extracted, maybe works.

previous answer:

| makeresults 
| eval _raw="
Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
----                ------   -------  -------------  ----------
DefaultAppPool      Started  V4.0     Integrated     OnDemand
.NET V4.5 Classic   Started  V4.0     Classic        OnDemand
.NET v4.5           Started  V4.0     Integrated     OnDemand
Monkey              Started  V4.0     Integrated     OnDemand"
`comment("this is your sample")`
`comment("from here, the logic")`
| multikv forceheader=1
| table Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats dc(Status) as Status_change by Name
| where Status_change > 1

Hi, how about this?
Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?
No, you should provide sample TEXT log.

Re: How to identify IIS Application Pool status changes

spayneort — Sat, 01 Feb 2020 17:53:22 GMT

Try adding | ConvertTo-Json -Compress to your PowerShell command.

Re: How to identify IIS Application Pool status changes

jsmithn — Tue, 04 Feb 2020 19:53:27 GMT

Your solution does identify situations where dc(Status) > 1, so it correctly finds situations when the service state has changed over a given period. But how would I still display the host, time of last change, and current status?

I can add "host" to my display parameter stats dc(Status) AS Status_change BY host,Name but if I add the other columns the dc count becomes invalid.

Re: How to identify IIS Application Pool status changes

to4kawa — Tue, 04 Feb 2020 21:17:38 GMT

how would I still display the host, time of last change, and current status?
There is no sample text log.
host? time? I can't see them.

Re: How to identify IIS Application Pool status changes

jsmithn — Tue, 04 Feb 2020 21:34:58 GMT

Aren't host and _time native parameters splunk includes as part of every event ingested? There is no logfile; the powershell script calls the Get-IISAppPool object and the output of that is directly ingested, which is the output I originally included.

I've modified your suggestion like this but it's kludgy:

It shows all the AppPools on a given host when I really only want to display the status of the AppPools where SC > 1. Ideally instead of showing the impacted AppPool twice, it would just show once, and include the time of the last change and the current status.

Re: How to identify IIS Application Pool status changes

to4kawa — Wed, 05 Feb 2020 09:03:10 GMT

my latest answer is checked?

Re: How to identify IIS Application Pool status changes

jsmithn — Wed, 30 Sep 2020 04:03:04 GMT

@to4kawa, thanks for the answer. I modified it slightly just for formatting purposes and to grab the "latest" instead of "last" event. Final query follows:
my_search
| multikv forceheader=1
| table _time host Name Status
| stats latest(_time) as _time latest(Status) as Status dc(Status) as SC by host Name
| where SC > 1
| table _time host Name Status

Re: How to identify IIS Application Pool status changes

to4kawa — Wed, 05 Feb 2020 21:54:46 GMT

I see, happy splunking 🙂

Re: How to identify IIS Application Pool status changes

alexlexxy — Mon, 03 Jan 2022 03:50:38 GMT

@jsmithn I have the same requirement at work, could you please post a walkthrough how you were able to get the fields such as status etc. The logs does not show these fields. Any help will be appreciated.