Create Log File in Real Time

[shangshin]

Hi,
I have log files sending from the remote server using the SplunkForwarder program. Is there a way from the splunk server to write the log file and have it stored on the splunk server's filesystem in real time? It will be like running unix command rsync from remote server to the splunk server.

Thanks in advance! 

[alacercogitatus · Accepted solution]

I believe that there is. I assume this is for some sort of compliance issue, we do a very similar thing but in reverse.

To answer your question, I am assuming a *nix environment. Splunk has the ability to send syslog events out, so I would setup syslog-ng or something similar on the indexers, and forward from splunk to syslog-ng and capture it that way. The full explanation for doing this (splunk side) is here: http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Forwarddatatothird-partysystemsd#Syslog_data

[alacercogitatus]

Depends on the system and program installed to handle the TCP traffic.

[shangshin]

That's exactly what I need!

Assuming I am using TCP to forward all data using the stanza below. How does the receiver, the non-splunk system, write the log file into file system? Is there a sample program? Many Thanks!!!

[tcpout:fastlane]
server = 10.1.1.35:6996
sendCookedData = false

[sdaniels]

Why do you want to do this? The data from your log files on the remote server will be stored in Splunk and you'll have access to them in real time.