topic Re: getting file logs in Dashboards & Visualizations

getting file logs

henrytran — Wed, 12 Aug 2020 20:59:25 GMT

Hello,

I am working on getting the logs into a dashboard. Files are sitting in the source a 2 minutes and will be moved to another server after 2 minutes time frame. my concern is if there is a script pull the logs within the time frame.

Thanks in advance

Henry

Re: getting file logs

gcusello — Thu, 13 Aug 2020 08:42:07 GMT

Hi @henrytran,

if you have a Universal Forwarder on the server where files are stored, it reads the logs every 30 second (by default but it's configurable), so your files are read before they move.

So you don't need a script.

A script could help if your cannot install a UF on that server, but this isn't a Splunk question, it's a Linux question.

Ciao.

Giuseppe

Re: getting file logs

henrytran — Thu, 13 Aug 2020 16:28:46 GMT

I am clarifying that how quick the Universal Forwarder reads the files? because I am concerning about 2 minutes limitation in reading 500 files or 50,000 files.

Thank you for responding to my questions.

Henry T

Re: getting file logs

gcusello — Thu, 13 Aug 2020 17:11:30 GMT

Hi @henrytran,

I think that an UF can read 50,000 linea in 2.5 minutes, but to be sure, you can do a test.

If your UF is too slow, you can copy the files in another folder, use it for reading and delete files after, but I don't think that's necessary.

Ciao.

Giuseppe

Re: getting file logs

henrytran — Fri, 14 Aug 2020 15:41:45 GMT

I appreciate it for your responses.