I have 4 VMs on a system for testing - RHEL 5.6, Win 7 64-bit Enterprise, Win Server 2008 R2, and Win XP Pro 32-bit.
I have configured Splunk server on the 2008 box.
The major issue I am seeing is -
With Snare Agent (free version is UDP, which I'm using for testing) - all clients send perfectly-formatted log data to the splunk server. Out of the box, everything just works perfectly.
Testing Splunk Universal Forwarder as a client, on a different port, as tcp (out of the box), I'm getting fragmented lines in the splunk server - some entries are one line, others two. Absolutely no indication of which machine sent the details. Line wrap IS enabled.
For some reason, Splunk configures the Universal Forwarder to send cooked (formatted) tcp data to the server. Splunk server shows it as a what appears to be escape sequences. Configuring the outputs.conf file to set cooked data from true to false fixes that.
But still, I cannot get the universal forwarder to send complete, usable data to any given entry of the splunk server.
I tried to also translate the snare agent config file lines to the most equivalent from the outputs.conf docs file splunk offers on their web site, but that didn't seem to do much.
What am I missing?
I would really like to have a tcp connection from client to server, ideally encrypted, and splunk server is an excellent product from what I can see. I presume splunk universal forwarder can do the job I want, it is a just a matter of figuring out how. Snare Agent can do it perfectly, but we'd need to buy it for tcp capability.
Any help on getting splunk to talk to splunk with full details of each log entry would be most appreciated.
That seems to have fixed it - at least getting properly formatted data from the Windows hosts - waiting on the Linux host to chime in.
Next, the splunktcp port is NOT showing activity (not showing up at all) in Splunk> Search > Sources
It would be nice to simply select the port and then run a query on that port for hits.
The individual windows hosts are listed with currently updates times.
Thanks again for any help with this.
and the splunktcp port will not show up with sources. the whole point is that the source will be the actual source of the data that the forwarder collections, not the (basically uninformative) port number of how it got to the splunk server.
And also, you may have to tweak some settings in props.conf on the indexer regarding line_breaking (which is splunk-speak for event-breaking, i.e. determining where one event ends and the next one starts.)