URL Monitoring issue

[jackin]

Hello,
We have a few URLs being monitored by a Splunk alert(query pasted below for reference) by making use of the "Website Monitoring" add on.

index=myindex sourcetype="web_ping"
[| inputlookup URL.csv]
| streamstats count by response_code url
| where count>=2 and response_code>=300
| eval Timestamp=strftime(_time ,"%d/%m/%Y %H:%M:%S"),Status="Failure"
| rename response_code as "HTTP Response Code" url as URL
| dedup URL
| table Timestamp "HTTP Response Code" URL Status

 Here the problem is 
we are receiving response_code and response_time fields as empty like below 

proxy_server="" title=abc.com timed_out=False proxy_port="" url=https://abc.com total_time="" request_time="" timeout=120 response_code="" proxy_type=http

can anyone suggest to resolve (troubleshooting steps) this issue.

[SinghK]

Thats an issue with that add-on. I think you need to highlight that to developer. Or you need to readjust the frequency it polls or reduce no of urls on the hf where addon is .

But I used to eval that empty code as 404 or url unreachable  and then you will have value instead of empty value.

[gcusello]

Hi @jackin,

what is the problem: aren't the missed fields extracted from logs or what else?

could you share some samples of your logs, bolding the missed fields?

Ciao.

Giuseppe

[jackin]

Hi @gcusello 

Like above snap some fileds are empty

[gcusello]

Hi @jackin,

the question is: are they empty because they aren't present or because there's a parsing error?

for this reason I asked a sample of your log, with the missing values.

Ciao.

Giuseppe

[jackin]

Hi @gcusello 

For ex: we are trying to access one of the url htttps://.... , so basically it is giving as HTTP Error 503 , But Splunk is showing HTTPP response code as null .

So, we can say that it's a parsing error

[gcusello]

Hi @jackin,

This means that there's a parsing error, if you could share a sample of your logs, I could help you to create another field extraction.

Ciao.

Giuseppe

[jackin]

Hi @gcusello 

Thanks for help

where it is available ? 

[gcusello]

Hi @jackin,

As I said, probably there's a parsing error in your TA, so the easiest way is to create a new field extraction.

But to do this, I need that you share a sample of your logs.

Ciao.

Giuseppe

[jackin]

Hi @gcusello 

I am asking which logs you need and where it is available 

[gcusello]

Hi @jackin,

you could send the logs that you have in the print screen you shared.

Ciao.

Giuseppe

[jackin]

Hi @gcusello 

PFB log

proxy_server="" title=internal-onyxquals-738401111.eu-west-1.elb.amazonaws.com timed_out=True proxy_port="" url=https://internal-onyxquals-738401111.eu-west-1.elb.amazonaws.com/view/adminLogin.jsp total_time="" request_time="" timeout=120 response_code="" proxy_type=http

[gcusello]

Hi @jackin,

if this is your row log, you cannot do anything because the values are missing in the source; it isn't a parsing error.

You could force response_code="504"  and response_time=_time-120, something like this:

| eval response_code=if(timed_out="True",504,response_code), response_time=if(timed_out="True",_time-120,response_code)

Ciao.

Giuseppe

[SinghK]

As I said earlier the add-on starts doing this when the no of tests increase on the hf it's installed. There is no fix to this just a work around only if you know your url is up when response code field is empty

|eval response_code = if ( response_code= "", "200" , response_code)

This will update empty response code as 200 else will keep what ever response code is there in actual event.