I have a splunk Indexer receiving data from Kiwi syslog installed on a Splunk Forwarder machine.
it also receives some specific data on UDP port 515 on the same server
Both works as expected!
now, I have configured port UDP 516 to receive another feed from security devices but it won't show up as collecting data on my indexer
I have checked the following:
1. Splunkd is running on port 516
2. syntax is rightly added in inputs.conf with out any errors
3. splunk list udp shows 515 & 516 on which splunk is listening
4. custom index to which data is forwarded is created on splunk indexer
any clue to fix this would be greatly appreciated!!
Why not use syslog defaults for everything and use Kiwi server as receiver and use the forwarder to read and forward the Kiwi logs? This way you were 'protected' against loosing syslog data when you restart your Splunk Forwarder.
I am still looking to troubleshoot it MuS, once I have found a fix, i'll mark this post as answered
got wireshark downloaded on my forwarder system to see if it can capture something on 516
BTW, all your help is much appreciated here MuS 🙂
Correction, thr is only 1 splunkd.exe which is listening on Forwarder.
514 is used is default syslog & 516 is whr i am trying to receive feed from another source
idea is to segregate data coming from various sources to this forwarder & then channelize them to different indexes
if i am not able to get the data from UDP 516 then i would go back to classic method of having everything received by Syslog & then read from it & further classify it via splunk methods
Thanks for your responses MuS!
yupp, I have tried receiving on a different UDP port on my kiwi syslog but it does not accept anything on 515 or 516 although my forwarder is able to receive on UDP 515.
splink list UDP was done on forwarder & it displays 514 & 515 as listening, even when i added 516 UDP, it shows to be listening on 516 but no data is collected on it
no, not if the difference is only a few minutes. Different approach, can your Kiwi Syslog server receive on second UDP Port? If so, try to receive data on UDP 515 & 516 and see if the Kiwi Syslog server gets your feed. btw I just checked your question...was your 'splunk list udp' done on the forwarder?
did search on all-time range+index(custom)+specific source but no results, also i checked the time stamp & it seems to lag by few minutes compare to the splunk forwarder's time. can difference in timestamp lead to index not receiving data?
did you check the time stamp of your source data feed? Maybe this is messed-up and you will find your data being indexed, but events have a wrong time stamp. Did you search all-time on your index and source?
nothing that i could find on 516 in splunkd.log 😞
I have ensured that my log source is able to make a successful connection with port 516 UDP on my forwarder
any further clues you may think of?
sorry my mess - this would be Port 516 on the forwarder not indexer! But nevertheless the source device must be able to reach UDP Port 516 on your forwarder. Do you see any errors in splunkd.log on the forwarder regarding UDP inputs or the source?
did not try that since, communication between indexer & forwarder is happening via TCP 9997
is it required that My Log source is directly able to communicate with my Indexer?
Thanks in advance for your response