I have a splunk Indexer receiving data from Kiwi syslog installed on a Splunk Forwarder machine.
it also receives some specific data on UDP port 515 on the same server
Both works as expected!
now, I have configured port UDP 516 to receive another feed from security devices but it won't show up as collecting data on my indexer
I have checked the following:
1. Splunkd is running on port 516
2. syntax is rightly added in inputs.conf with out any errors
3. splunk list udp shows 515 & 516 on which splunk is listening
4. custom index to which data is forwarded is created on splunk indexer
any clue to fix this would be greatly appreciated!!
did not try that since, communication between indexer & forwarder is happening via TCP 9997
is it required that My Log source is directly able to communicate with my Indexer?
Thanks in advance for your response
sorry my mess - this would be Port 516 on the forwarder not indexer! But nevertheless the source device must be able to reach UDP Port 516 on your forwarder. Do you see any errors in splunkd.log on the forwarder regarding UDP inputs or the source?
nothing that i could find on 516 in splunkd.log 😞
I have ensured that my log source is able to make a successful connection with port 516 UDP on my forwarder
any further clues you may think of?
did you check the time stamp of your source data feed? Maybe this is messed-up and you will find your data being indexed, but events have a wrong time stamp. Did you search all-time on your index and source?
did search on all-time range+index(custom)+specific source but no results, also i checked the time stamp & it seems to lag by few minutes compare to the splunk forwarder's time. can difference in timestamp lead to index not receiving data?
no, not if the difference is only a few minutes. Different approach, can your Kiwi Syslog server receive on second UDP Port? If so, try to receive data on UDP 515 & 516 and see if the Kiwi Syslog server gets your feed. btw I just checked your question...was your 'splunk list udp' done on the forwarder?
yupp, I have tried receiving on a different UDP port on my kiwi syslog but it does not accept anything on 515 or 516 although my forwarder is able to receive on UDP 515.
splink list UDP was done on forwarder & it displays 514 & 515 as listening, even when i added 516 UDP, it shows to be listening on 516 but no data is collected on it
Why not use syslog defaults for everything and use Kiwi server as receiver and use the forwarder to read and forward the Kiwi logs? This way you were 'protected' against loosing syslog data when you restart your Splunk Forwarder.