topic Re: How to get http error codes count in Security

How to get http error codes count

cloud111 — Tue, 30 May 2017 10:00:23 GMT

Hi,

I have the following log:

01.01.01.56 - - [20/May/2016:09:22:44 +0000] "GET /parking/js/node.js HTTP/1.1" 302 -

01.01.01.56 - - [20/May/2016:06:44:44 +0000] "GET /outside/js/node.js HTTP/1.1" 404 -

How do i run a search to extract all the different HTTP error codes other then 200 and graph the results for example

http 302 = 130
http 404 = 90
In the end i want it to be displayed as a bar gragh

I used:
sourcetype=tomcat 400 OR 401 OR 403 OR 404 OR 502 OR 503

Re: How to get http error codes count

niketn — Tue, 30 May 2017 14:37:16 GMT

Have you checked in verbose mode whether http status field is getting extracted for your tomcat sourcetype or not? If it is not then ideally you should created a Field Extraction for 304, 404 using Splunk's Interactive Field Extraction through Extract Fields option in the Search Menu. Refer to documentation http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ExtractfieldsinteractivelywithIFX

Otherwise temporarily you can create Field extraction directly in your SPL using the rex command if you are aware of Regular Expressions. You might have to try with different types of sample to make sure Regular Expression is universal.

sourcetype=tomcat 400 OR 401 OR 403 OR 404 OR 502 OR 503
| rex field=_raw "(?ms)^(?:[^ \\n]*){8}(?P<status>\\d+)"
| stats count by status

PS: Above rex is based on Data Sample Provided. Once you have tested regular expression using rex, you should create a Field Extraction through the Regular Expression.

Another not so clean option would be to use searchmatch

sourcetype=tomcat 400 OR 401 OR 403 OR 404 OR 502 OR 503
| eval status=case(searchmatch("400"),400,searchmatch("401"),401,searchmatch("403"),403,searchmatch("404"),404,searchmatch("502"),502,searchmatch("503"),503,true(),"Unknown")
| stats count by status

Do check out Splunk Add On for Tomcat on Splunkbase, so that Tomcat data is processed as per CIM.

Re: How to get http error codes count

anthonymelita — Tue, 30 May 2017 20:55:11 GMT

if you want to shorten your search string and get anything which isn't 200 you can also just use
status!=200

Re: How to get http error codes count

WalshyB — Wed, 31 May 2017 13:38:12 GMT

Maybe this? To stop you having to write out every status code except 200.

anything other than 200 -
sourcetype=tomcat | insert regex to extract status here| where status!=200 | stats count by status

or if you create status as a search time extraction-
sourcetype=tomcat status!=200 | stats count by status

To have a more accurate "error" reading, use >399