topic Re: Create Log File in Real Time in Splunk Search

Create Log File in Real Time

shangshin — Wed, 14 Nov 2012 13:43:48 GMT

Hi,
I have log files sending from the remote server using the SplunkForwarder program. Is there a way from the splunk server to write the log file and have it stored on the splunk server's filesystem in real time? It will be like running unix command rsync from remote server to the splunk server.

Thanks in advance!

Re: Create Log File in Real Time

sdaniels — Wed, 14 Nov 2012 13:51:22 GMT

Why do you want to do this? The data from your log files on the remote server will be stored in Splunk and you'll have access to them in real time.

Re: Create Log File in Real Time

alacercogitatus — Wed, 14 Nov 2012 14:01:20 GMT

I believe that there is. I assume this is for some sort of compliance issue, we do a very similar thing but in reverse.

To answer your question, I am assuming a *nix environment. Splunk has the ability to send syslog events out, so I would setup syslog-ng or something similar on the indexers, and forward from splunk to syslog-ng and capture it that way. The full explanation for doing this (splunk side) is here: http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Forwarddatatothird-partysystemsd#Syslog_data

Re: Create Log File in Real Time

shangshin — Wed, 14 Nov 2012 16:36:31 GMT

That's exactly what I need!

Assuming I am using TCP to forward all data using the stanza below. How does the receiver, the non-splunk system, write the log file into file system? Is there a sample program? Many Thanks!!!

[tcpout:fastlane]
server = 10.1.1.35:6996
sendCookedData = false

Re: Create Log File in Real Time

alacercogitatus — Wed, 14 Nov 2012 17:07:43 GMT

Depends on the system and program installed to handle the TCP traffic.