Splunk Search

How to identify IIS Application Pool status changes

Path Finder

I know similar questions have been asked a number of times but trying to follow the suggestions given I still cannot get it to work. Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?

I want the search to show start/stop status changes which we'll use in an actionable alert. Output items should include host, apppool, time of last change, and current status.

The Get-IISAppPool output (ingested into Splunk) looks like this:
alt text

0 Karma
1 Solution

Ultra Champion

recommend:

your_search
| multikv forceheader=1
| table _time host Name  Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats last(_time) as _time last(Status) as current_Status dc(Status) as Status_change by Name host
| where Status_change > 1

If all field is extracted, maybe works.


previous answer:

| makeresults 
| eval _raw="
Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
----                ------   -------  -------------  ----------
DefaultAppPool      Started  V4.0     Integrated     OnDemand
.NET V4.5 Classic   Started  V4.0     Classic        OnDemand
.NET v4.5           Started  V4.0     Integrated     OnDemand
Monkey              Started  V4.0     Integrated     OnDemand"
`comment("this is your sample")`
`comment("from here, the logic")`
| multikv forceheader=1
| table Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats dc(Status) as Status_change by Name
| where Status_change > 1

Hi, how about this?
Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?
No, you should provide sample TEXT log.

View solution in original post

0 Karma

Path Finder

@to4kawa, thanks for the answer. I modified it slightly just for formatting purposes and to grab the "latest" instead of "last" event. Final query follows:
my_search
| multikv forceheader=1
| table _time host Name Status
| stats latest(_time) as _time latest(Status) as Status dc(Status) as SC by host Name
| where SC > 1
| table _time host Name Status

0 Karma

Ultra Champion

I see, happy splunking 🙂

0 Karma

Contributor

Try adding | ConvertTo-Json -Compress to your PowerShell command.

0 Karma

Ultra Champion

recommend:

your_search
| multikv forceheader=1
| table _time host Name  Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats last(_time) as _time last(Status) as current_Status dc(Status) as Status_change by Name host
| where Status_change > 1

If all field is extracted, maybe works.


previous answer:

| makeresults 
| eval _raw="
Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
----                ------   -------  -------------  ----------
DefaultAppPool      Started  V4.0     Integrated     OnDemand
.NET V4.5 Classic   Started  V4.0     Classic        OnDemand
.NET v4.5           Started  V4.0     Integrated     OnDemand
Monkey              Started  V4.0     Integrated     OnDemand"
`comment("this is your sample")`
`comment("from here, the logic")`
| multikv forceheader=1
| table Name                Status   CLR_ver  Pipeline_Mode  Start_Mode
| stats dc(Status) as Status_change by Name
| where Status_change > 1

Hi, how about this?
Perhaps I need to modify the output of the powershell command to create a key/value pair for name+status?
No, you should provide sample TEXT log.

View solution in original post

0 Karma

Path Finder

Your solution does identify situations where dc(Status) > 1, so it correctly finds situations when the service state has changed over a given period. But how would I still display the host, time of last change, and current status?

I can add "host" to my display parameter stats dc(Status) AS Status_change BY host,Name but if I add the other columns the dc count becomes invalid.

0 Karma

Ultra Champion

how would I still display the host, time of last change, and current status?
There is no sample text log.
host? time? I can't see them.

0 Karma

Path Finder

Aren't host and _time native parameters splunk includes as part of every event ingested? There is no logfile; the powershell script calls the Get-IISAppPool object and the output of that is directly ingested, which is the output I originally included.

I've modified your suggestion like this but it's kludgy:

host=xyz sourcetype=IISAppPool earliest=-5m | multikv forceheader=1
| table host _time Name Status
| append
[ search host=xyz sourcetype=IISAppPool earliest=1h| multikv forceheader=1
| stats dc(Status) AS SC BY host,Name
| where SC > 1 ]
| table host _time Name Status

It shows all the AppPools on a given host when I really only want to display the status of the AppPools where SC > 1. Ideally instead of showing the impacted AppPool twice, it would just show once, and include the time of the last change and the current status.

0 Karma

Ultra Champion

my latest answer is checked?

0 Karma