topic Re: Monitor logs from remote server without installation of UF in Installation

How to monitor logs from remote server without installation of Universal Forwarder?

ips_mandar — Thu, 11 Jun 2020 21:35:39 GMT

I have one server A which is network connected to Server B where Splunk is installed, and I want to monitor a few folders present in Server A without installation of universal forwarder due to some restrictions.

I am on Windows OS and I can browse to that folder and can read folder files from file explorer by following path- \\abcstorage\xyz

Now to monitor this in Splunk what path needs to be mentioned in splunk inputs.conf?I tried below inputs, but am still unable to monitor.

[monitor:////abcstorage\xyz\*.zip]
disabled = false
index = xyz
sourcetype = abc
ignoreOlderThan = 1d

Thanks.

Re: Monitor logs from remote server without installation of UF

adonio — Wed, 10 Jul 2019 12:48:25 GMT

check permission, can the forwarder read the file in the path?

Re: Monitor logs from remote server without installation of UF

gcusello — Wed, 10 Jul 2019 13:27:53 GMT

Hi ips_mandar,
if you share your path with the E: drive use something like this

 [monitor://E:\abcstorage\xyz\*.zip]
 disabled = false
 index = xyz
 sourcetype = abc
 ignoreOlderThan = 1d

As suggested by Adonio, check permissions to be sure that forwarder can read the files.

Bye.
Giuseppe

Re: Monitor logs from remote server without installation of UF

ips_mandar — Wed, 10 Jul 2019 14:50:17 GMT

Hi @adonio @gcusello ,
System has permission and I can view all files from network drive..it is network shared drive and it is not present in same system where splunk is installed. Network>abcstorage>xyz

Re: Monitor logs from remote server without installation of UF

FrankVl — Wed, 10 Jul 2019 15:13:12 GMT

I believe the correct syntax is [monitor://\\abcstorage\xyz\*.zip].

Alternatively, you can mount the network drive under a local drive letter (or nowadays even as a folder inside the local filesystem I believe?) and then use something like @gcusello mentioned.

Re: Monitor logs from remote server without installation of UF

ips_mandar — Wed, 10 Jul 2019 16:00:13 GMT

Thanks a lot @FrankVI
If I create local drive as suggested then does it will impact performance?
I will test and keep you posted.
thanks.

Re: Monitor logs from remote server without installation of UF

ips_mandar — Wed, 10 Jul 2019 17:49:43 GMT

Hi @FrankVI @gcusello I tried networkdrive like [monitor://Z:\xyz\*.zip] and [monitor://\\abcstorage\xyz\*.zip]
but none of them is working.

Re: Monitor logs from remote server without installation of UF

FrankVl — Thu, 11 Jul 2019 06:25:51 GMT

Any clues in splunkd.log? Is it trying to start monitoring that path?

Re: Monitor logs from remote server without installation of UF

ips_mandar — Thu, 11 Jul 2019 06:49:48 GMT

@FrankVI @gcusello
Yesterday I kept [monitor://\\abcstorage\xyz\*.zip] this stanza in inputs.conf and till yesterday no data in indexed and when today I checked again and I see that data being indexed which are newly inserted yesterday night probably(its modified date is 10 July).
although there was many files from 9 July as well but none of them got indexed yesterday, although I set ignoreOlderThan = 1d so it will not index these file today ,that's Ok but why they not indexed yesterday?

Re: Monitor logs from remote server without installation of UF

gcusello — Thu, 11 Jul 2019 06:59:34 GMT

Hi ips_mandar,
check you date format: if you have dd/mm/yyy probably your Splunk inverted months and days, so you can find your yesterday logs in october.
In this case, you have to fix the timestamp format in props.conf.

Bye.
Giuseppe

Re: Monitor logs from remote server without installation of UF

ips_mandar — Thu, 11 Jul 2019 07:09:35 GMT

there won't be any timestamp issue since yesterday's file got indexed since yesterday's file contain two days back data and it is indexed as per timestamp in the events.
Thanks.

Re: Monitor logs from remote server without installation of UF

FrankVl — Thu, 11 Jul 2019 07:20:37 GMT

Good to hear that it started working 🙂

Yesterday was the 10th right? So files from 9th may have already fallen outside scope? I'd just keep an eye on if it now continues working consistently.

Re: Monitor logs from remote server without installation of UF

ips_mandar — Thu, 11 Jul 2019 07:36:31 GMT

Yes Yesterday was 10th and these 10th July files are indexed in splunk ..it may be due to last 24 hours when I created inputs.conf due to which no files got indexed from 9th.
Now I created another input monitoring another folder and included ignoreOlderThan = 1d
But it is not indexing 10 July Files..

Re: Monitor logs from remote server without installation of UF

ips_mandar — Thu, 11 Jul 2019 07:46:57 GMT

I checked modification time of 10 July file which is not being indexed is 10 July 2:22 AM
and Current time 11 July 1:12 PM it seems due to 24 hours are already past it will not index these files 🙂
one question my system time zone is IST and the server whose files are monitored is in different timezone but when I am browsing to that folder via network shared server the modification time it will show will according to my server timezone?

Re: Monitor logs from remote server without installation of UF

FrankVl — Thu, 11 Jul 2019 07:49:31 GMT

I do know from experience that such remote share monitoring is sometimes quite slow to get going (especially if the forwarder is still busy scanning / ingesting other remote folders).

What is the exact creation/modification date/time on those files and what is the system time on these systems? Also not sure how 1d is interpreted, it may not be the same as 24h, it might simply check the dates only.

Re: Monitor logs from remote server without installation of UF

FrankVl — Thu, 11 Jul 2019 07:53:48 GMT

Good question on the time zones. Not sure to be honest. If you have write permissions you could test that.

Re: Monitor logs from remote server without installation of UF

ips_mandar — Thu, 11 Jul 2019 08:53:50 GMT

I checked one file and details are-

Created-Today, ‎July ‎11, ‎2019, ‏‎11 hours ago
Modified-Yesterday, ‎July ‎10, ‎2019, ‏‎2:22:52 AM

isn't it strange that modified is 10 July and Created is 11 July?