Splunk running on my linux server is only showing me events from my local subnet, what is going on?

[jbsplunk]

Splunk is running on my Linux box, and everything appears to be operating almost correctly. I have data coming in from a LogLogic box via UDP, and that data is being spoofed in such a way to make it appear as though the data is coming from the originating sources, and not the LogLogic device which sits on the same network as Splunk.

I am getting events, however, these events are only from my local subnet. When I change the subnet mask for the interface the events are arriving on, I can see events that are included in the range allowed by that subnet mask.

For example, If I use a /24 mask, I see events from the local network. If I use a /18, I see events from the Class B network. However, I do not see events that are from outside of the local network.

How can I resolve this?

[jbsplunk · Accepted solution]

When using IPv4 packet forwarding, you will also get the rp_filter, which automatically rejects incoming packets if the routing table entry for their source address doesn't match the network interface they're arriving on. This has security advantages because it prevents the so-called IP spoofing, however it can pose problems if you use asymmetric routing (packets from you to a host take a different path than packets from that host to you) or if you operate a non-routing host which has several IP addresses on different interfaces.

You can turn this off by editing /etc/sysctl.conf, set net.ipv4.conf.default.rp_filter = 0