Log ingestion with line level delete

bworrellZP — Wed, 05 Jun 2024 00:16:40 GMT

This evening decided to setup a test Splunk box in my lab to goof around with. Been a while since I have done this part of the process. (Work cluster is up and going and has been for years now). As I was looking at my local test box, I noticed the hard drive was not likely the best size to use.

So since I have a syslog server running on this as well, and I am pulling those files into Splunk (Splunk will not always be running, hence not sending data direct to Splunk), wanted to try doing a line level destructive read.

I did see where others were using a monitor and deleting a file on ingestion, but did not see if line level was being done.

So, question is, has anyone done that, and if so, do you have some hints or pointers?

Thanks

Re: Log ingestion with line level delete

gcusello — Wed, 05 Jun 2024 06:15:13 GMT

Hi @bworrellZP,

you could use (only in lab) the syslog network input that doesn't write on disk.

Otherwise, use rsyslog, writing syslog on disk and then read these logs using the batch command, instead monitor, in the inputs.conf.

For more infos see at https://docs.splunk.com/Documentation/Splunk/9.2.1/Admin/Inputsconf

In this way logs are deleted soon after ingestion.

Ciao.

Giuseppe

topic Re: Log ingestion with line level delete in Getting Data In

Log ingestion with line level delete

Re: Log ingestion with line level delete