Splunk Search

How to get the number of errors for each application ?

lsy9891
Engager

Hi,

I'm new to Splunk and so far I've managed to get the number of errors but I do not know for which application? I know we need an index and host but what exactly should be the index? Here's what I tried.

 error OR failed OR severe OR ( sourcetype=access_* ( 404 OR 500 OR 503 ) ) | stats count by error
0 Karma

woodcock
Esteemed Legend

You can get a better view with a search like this:

(index=* OR index=_*) AND (error OR failed OR severe OR ( sourcetype=access_* ( 404 OR 500 OR 503 ) ))
| stats first(_raw) last(_raw) values(host) count BY punct sourcetype index
0 Karma

sixcorners
Loves-to-Learn Lots

In the list of fields on the left side you can click them and click on the "Top Values" report to get a breakdown of the top counts for the field you select. It basically just adds "| top limit=20 x" to the end of your query where x is the field you clicked.
If I want a count of how many errors are coming from each app I would use that to make this query:
severity=ERROR| top limit=20 source

0 Karma

Sukisen1981
Champion

can you paste what values your index,source and surcetype fields contain?
Ideally you should consider naming your indexes after your applications and let them pick different folders, app logs whatever it is from the default source n sourcetypes
Host will typically be your server name , like xxx_prod_1.0 or something like that

0 Karma

lsy9891
Engager

Hi,may I know how to get the index, source and sourcetype fields of the app?

0 Karma

Sukisen1981
Champion

hi @lsy9891
you are using sourcetype=access_* ( 404 OR 500 OR 503 ), which means you are accessing all sourcetypes starting with access
you can use a query like this, for determining your indexes

index="*" 
|stats values(source),values(sourcetype) by index

Run this for last 15 mins (DO NOT RUN FOR ALL TIME / 24 HRS) and you should get all your index, source n sourcetypes listed. You need to choose the relevant ones.

0 Karma

lsy9891
Engager

Hi, I've managed to get the number of errors however, I tried to extract all the application names using regex but some are omitted. This is the rex I used:

rex field=WindowsIdentity "(?P\w+.\w+)". The field I'm supposed to extract is anything after the ISS APPOOL and just before the .monster. For example,

IIS APPPOOL\ jobs.monster.com
IIS APPPOOL\ *hiring.channels *.monster.com_jcm
IIS APPPOOL\ *wwwcs.channels *.monster.com

0 Karma

Sukisen1981
Champion

hi @lsy9891
You have posted a separate question for the rex and that has been answered as well, you can extract these values using the rex below as well | rex field=WindowsIdentity "\\\+(?<Description>.*?)\.+monster"
can you give examples where rex is failing?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...