I am trying to use the Web datamodel in Splunk ES. This datamodel seems to be missing the distinction between inbound web traffic and outbound web traffic. In fact it seems mostly to focused on inbound web requests. Am I missing something?
The distinction is important as inbound web requests are more indicative of external entity attack activity
while outbound web requests are more indicative of compromised systems.
I think that I can address the difference by my knowledge that different systems are the source for model inbound and outbound web request data, but it seems that this should be abstracted into the model somehow.
1) Can Splunk consider updating the Web datamodel to include some notion of inbound vs outbound?
2) How are other users of the Web datamodel dealing with this?
The Web data model has a child object of "Proxy" which is usually used for outbound proxy events (think Bluecoat, Websense, etc). This would mean that your "normal" proxy data would be tagged as "web" and "proxy" whereas you reverse proxy/other web data (F5, Apache, etc.) would only be tagged as "web". This would allow you to filter results in tstats query on the "nodename" field.
Interesting. I guess that "Proxy" makes sense since one might have to correlate those logs to their pre-proxied source. Any place with use-cases for proxied and non-proxied traffic would have to be careful to account for both
In the end, I am seeing that all that Splunk offers is the ability for us to infer inbound vs outbound based on other properties. Not really sure how to help Splunk see that In/Out should really be part of the data model.
I see this long outstanding question that I previously asked. I have been addressing this though information implied by the fields. At least in my environment, inbound Web events are sourced from the web servers so I can count on fields that vary between web server logs and the firewall logs that handle the outbound requests
Web.action=firewall actions (allow/deny) vs HTTP methods
web server logs generally do not log destination info (since the user has already arrived)
I have not, but could use these implying values as the basis for a tag, so if Web.dest=unknown, I could tag it "inbound"
Otherwise I might tag it "outbound".... but that doesn't quite work since the firewall also routes traffic to the web servers. So "outbound" would have to be based on what I know of our network traffic (that it is sourced from inside our network, going to outside of our network and that the firewall categorizes the traffic as web traffic.)