topic Re: Unable to count error events by source IP in Getting Data In

Unable to count error events by source IP

fclarkgeek — Fri, 13 Sep 2013 20:47:51 GMT

Granted I am new to splunk, and while I am utilizing the tutorials and help, it seems that I can not get something as simple as a error by IP report to work.

I have loaded data into splunk, and I can search the data and find what I want - quite simply, errors as reported in an Apache error log. What I want to do is show a count of errors by source IP address. For example the line in the log reads like this ( where xx is the source IP address ) :
[Fri Aug 22 16:59:01 2013] [error] [client xx.xx.xx.xx] File does not exist: /home/file.jpg

It seems I can't index by the source IP , so I attempted to to create a regex to extract that value. I was able to do so using the interactive field extractor. It was able to generate a regex pattern, which appeared to work properly.

Unfortunately , when I click on the TEST button , a window pops up for a few moments with some information on testing the regex, however it disappears after a few seconds before I can read it. I can see the buttons " cancel " and " test " , but again the window closes before I can click on either.

The same happens when I try to save the generated regex. I click on the "save" button and a window pops up requesting a field name, but again it closes before any information can be given.

This is an incredibly simple thing to do : Show me a count of errors, by source IP address. Can anyone please direct me to where I am making a mistake?

Re: Unable to count error events by source IP

lguinn2 — Fri, 13 Sep 2013 21:04:27 GMT

It sounds like something is wrong with your install or with the Interactive Field Extractor. Here is my suggestion: don't click the Test button. It sounds like a dumb suggestion, but it really isn't. Just save the field extraction and try it -- if you don't like it, you can just delete it and try again. You don't truly need the Test button.

Field extractions in Splunk are dynamic. You can create and delete them at will, without affecting the underlying data store.

I would definitely create the field extraction. But if you want, you can create an on-the-fly field like this:

source=yourlogfile "File does not exist" 
| rex "\[client\s(?<src_ip>\S+?)\]" 
| stats count by src_ip

(I hope I got the regular expression right!)

Re: Unable to count error events by source IP

fclarkgeek — Fri, 13 Sep 2013 21:09:14 GMT

Unfortunately I have tried saving the field extraction, and as I indicated the result was the same : a window pops up for a moment asking me to name the field, but disappears before I can input anything and the field is not saved.

I'm going to try your ' on the fly ' method now. I'm familiar with regex, but I need to ask , is in your regex considered a field name ?

Also , I am not looking specfically for " File does not exist " , I am looking for any error, so I should be able to replace that with "error" as I did in my initial search, correct?

Thank you for your assistance!

Re: Unable to count error events by source IP

fclarkgeek — Fri, 13 Sep 2013 21:20:12 GMT

I tried what you suggested , slightly modified :
"error" | rex "\[client\s(?<src_ip>\S+?)\]" | stats count by src_ip

It came back with no results. When I used the inspector it indicated that " stats count by src_ip " generated no results.

To ensure the regex was right ( I believed it was! ) I redid the query using :

"error" | rex "\[client\s(?<src_ip>\S+?)\]"

and it DID find results - so I believe your regex is correct, but still something is not quite working.

Re: Unable to count error events by source IP

lguinn2 — Fri, 13 Sep 2013 22:25:00 GMT

"error" | rex "\[client\s(?<src_ip>\S+?)\]"

gets results, take a look at the fields sidebar (on the left) and see if you can find the src_ip field. I believe that the field is probably not right, and that is why the stats command is not returning results.