Installation

How to monitor logs from remote server without installation of Universal Forwarder?

ips_mandar
Builder

I have one server A which is network connected to Server B where Splunk is installed, and I want to monitor a few folders present in Server A without installation of universal forwarder due to some restrictions.

I am on Windows OS and I can browse to that folder and can read folder files from file explorer by following path- \\abcstorage\xyz

Now to monitor this in Splunk what path needs to be mentioned in splunk inputs.conf?I tried below inputs, but am still unable to monitor.

[monitor:////abcstorage\xyz\*.zip]
disabled = false
index = xyz
sourcetype = abc
ignoreOlderThan = 1d

Thanks.

Labels (2)
0 Karma

FrankVl
Ultra Champion

I believe the correct syntax is [monitor://\\abcstorage\xyz\*.zip].

Alternatively, you can mount the network drive under a local drive letter (or nowadays even as a folder inside the local filesystem I believe?) and then use something like @gcusello mentioned.

ips_mandar
Builder

Thanks a lot @FrankVI
If I create local drive as suggested then does it will impact performance?
I will test and keep you posted.
thanks.

0 Karma

ips_mandar
Builder

Hi @FrankVI @gcusello I tried networkdrive like [monitor://Z:\xyz\*.zip] and [monitor://\\abcstorage\xyz\*.zip]
but none of them is working.

0 Karma

ips_mandar
Builder

@FrankVI @gcusello
Yesterday I kept [monitor://\\abcstorage\xyz\*.zip] this stanza in inputs.conf and till yesterday no data in indexed and when today I checked again and I see that data being indexed which are newly inserted yesterday night probably(its modified date is 10 July).
although there was many files from 9 July as well but none of them got indexed yesterday, although I set ignoreOlderThan = 1d so it will not index these file today ,that's Ok but why they not indexed yesterday?

0 Karma

FrankVl
Ultra Champion

Good to hear that it started working 🙂

Yesterday was the 10th right? So files from 9th may have already fallen outside scope? I'd just keep an eye on if it now continues working consistently.

0 Karma

ips_mandar
Builder

Yes Yesterday was 10th and these 10th July files are indexed in splunk ..it may be due to last 24 hours when I created inputs.conf due to which no files got indexed from 9th.
Now I created another input monitoring another folder and included ignoreOlderThan = 1d
But it is not indexing 10 July Files..

0 Karma

FrankVl
Ultra Champion

I do know from experience that such remote share monitoring is sometimes quite slow to get going (especially if the forwarder is still busy scanning / ingesting other remote folders).

What is the exact creation/modification date/time on those files and what is the system time on these systems? Also not sure how 1d is interpreted, it may not be the same as 24h, it might simply check the dates only.

0 Karma

ips_mandar
Builder

I checked one file and details are-

Created-Today, ‎July ‎11, ‎2019, ‏‎11 hours ago
Modified-Yesterday, ‎July ‎10, ‎2019, ‏‎2:22:52 AM

isn't it strange that modified is 10 July and Created is 11 July?

0 Karma

ips_mandar
Builder

I checked modification time of 10 July file which is not being indexed is 10 July 2:22 AM
and Current time 11 July 1:12 PM it seems due to 24 hours are already past it will not index these files 🙂
one question my system time zone is IST and the server whose files are monitored is in different timezone but when I am browsing to that folder via network shared server the modification time it will show will according to my server timezone?

0 Karma

FrankVl
Ultra Champion

Good question on the time zones. Not sure to be honest. If you have write permissions you could test that.

0 Karma

FrankVl
Ultra Champion

Any clues in splunkd.log? Is it trying to start monitoring that path?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi ips_mandar,
if you share your path with the E: drive use something like this

 [monitor://E:\abcstorage\xyz\*.zip]
 disabled = false
 index = xyz
 sourcetype = abc
 ignoreOlderThan = 1d

As suggested by Adonio, check permissions to be sure that forwarder can read the files.

Bye.
Giuseppe

0 Karma

ips_mandar
Builder

Hi @adonio @gcusello ,
System has permission and I can view all files from network drive..it is network shared drive and it is not present in same system where splunk is installed. Network>abcstorage>xyz

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi ips_mandar,
check you date format: if you have dd/mm/yyy probably your Splunk inverted months and days, so you can find your yesterday logs in october.
In this case, you have to fix the timestamp format in props.conf.

Bye.
Giuseppe

0 Karma

ips_mandar
Builder

there won't be any timestamp issue since yesterday's file got indexed since yesterday's file contain two days back data and it is indexed as per timestamp in the events.
Thanks.

0 Karma

adonio
Ultra Champion

check permission, can the forwarder read the file in the path?

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...