Checking response type?

[asleeis]

Hi,

I acknowledge I may just be missing something here, but why are search engines coming up with scores, like "8". According to ProjectHoneyPot, I thought those were not actually "threatscores", but in the case of type "0", were used to identify the search engine or something like that. For type "1" or above, threatscore applies.

hxxp://www.projecthoneypot.org/httpbl_api.php

Any chance this could be updated so that the search engines don't get caught up in the actual threatscore data? Or perhaps offer more variables as an alternative to allow for a little more interpretation by users like me, in my own Splunk queries.

Thanks,
-Alex

[Matthias_BY]

Hello Alex,

i did have time to fix it. now it should show the IP's of google bots etc. with threatscore 0.

best regards
Matthias

[asleeis · Accepted solution]

Hi Matthias,

I don't think my concern was fully understood. I do understand that the third octet is the threatscore... however, if you read further in their documentation, that is NOT true when the forth octet is "0". In that case, a forth octet of 0 indicates the IP is a search engine IP address, and the third octet ONLY identifies which search engine. In that case, it is not actually a threatscore.

I do think that ProjectHoneyPot confuses things by adding information into the result that isn't entirely related to the idea of "threat detection"... but that's the standard they went with (which is documented as such in the link I referenced, at least). You need to scroll down about 3/4 of the Project Honeypot API documentation page to see the details about Search Engines.

I guess the implication is that if the IP belongs to a valid search engine, it is not a threat, and thus the threatscore would be 0.

In your code, I would suggest that if the forth octet is 0 in the response, to set the threatscore to 0 and ignore the 3rd octet value from the response (which is only an identifier for a search engine). This will help make the results from IP Reputation more accurate and more properly interpreting the Project Honeypot response.

Thanks for the responses!

[Matthias_BY]

i understand. You're right and the first one who recognizes that.

i need to think about how to fix it - your idea is already good. but i think to add the fourth octet as indicator - the community will have another "feature" they can work on and then modifying the splunk search if needed to only report on the given fields or e.g. remove the search engines… never less most won't recognize because we are lazy to read documentation… 😉

thanks a lot again for your response.

happy new year!
br
matthias

[asleeis]

Here's an excerpt from the API documentation about search engines:

The third octet (1 in the example above) is a serial number identifier to a particular search engine. The list of serial numbers corresponding with each listed search engine can be found below.

The fourth octet (0 in the example above) is the type identifier. Because it is zero it means that this particular IP belongs to a search engine. Remember that the rules described in this section only apply if the fourth octet is a zero.

[Matthias_BY]

Hello Alex,

i'm glad that you ask your question here. At the moment the enriched field is only the threatscore - so the third octet from the project honeypot response.

The third octet (5 in the example above) represents a threat score for IP. This score is assigned internally by Project Honey Pot based on a number of factors including the number of honey pots the IP has been seen visiting, the damage done during those visits (email addresses harvested or forms posted to), etc. The range of the score is from 0 to 255, where 255 is extremely threatening and 0 indicates no threat score has been assigned. In the example above, the IP queried has a threat score of "5", which is relatively low. While a rough and imperfect measure, this value may be useful in helping you assess the threat posed by a visitor to your site.

The forth octet is within the lookup script - but currently i do not forward them to Splunk as an enriched field. first i thought more about the use case that you have a bunch of activities (maybe logons etc.) with external IP's and you might want to find some "malicious" threats and there next to iplocation by country etc. an good indicator is to lookup how "bad" the ip is that tried this and investigate those with a bad scoring first.

But great if you have an use case where the type of visitor is important for you.

Any chance this could be updated so that the search engines don't get caught up in the actual threatscore data?  Or perhaps offer more variables as an alternative to allow for a little more interpretation by users like me, in my own Splunk queries.

Yes you have a chance. Let me know if i did understand you correctly that you want the forth octet of the response in a field like "visitor_type" with the information of the visitor type.

Br
Matthias

[asleeis]

My question is about the app for splunk and whether I'm not understanding it right or if it really is needing an update. I'm not sure now that is inappropriate for the forum that is supposedly here to discuss and ask questions about the apps designed for splunk. I didn't realize all questions had to be ABOUT splunk, as opposed to being about the app.

I'll reach out to the author directly, but I still think a post for others to read when they see the same erroneous threatscores in their query results with this Splunk app is very relevant to the forums for the apps.

[rsennett_splunk]

You are absolutely right... I apologize for the confusing angle of my answer... I had pasted a couple of versions together and ended up with a mismatched set. I've also sent an email to the author via his internal address as it seems your question has been sitting unanswered and mostly unviewed for a couple of weeks. I thought you might want some attention, although I'm largely unfamiliar with the app itself. Best of luck with your inquiries.

[rsennett_splunk]

Since you are asking for updates to the IP Reputation app you might want to inquire directly to the author, who's contact information is here:

If you have questions about projecthoneypot itself you would have to direct it to the project org here

It isn't completely clear as to whether you are saying that the issue is with the rules as they're being interpreted by the app that seem incorrect, or whether project honeypot seems to be skewing the data incorrectly - do you see where there might be errors or perhaps have been a change to the rules somewhere?

If you don't actually have a Splunk related question this isn't really the forum for your inquiry... and you may have better luck directly connecting with Mattius.