topic Re: Windows - How to monitor XML files within a sub-directory in Getting Data In

Windows - How to monitor XML files within a sub-directory

splunker12er — Wed, 27 Aug 2014 08:58:45 GMT

I want to monitor XML files residing inside sub-directories.

Files inside Path :

D:\Roll\DIP\SessionLogs\35\1.xml
D:\Roll\DIP\SessionLogs\35\2.xml
D:\Roll\DIP\SessionLogs\35\3.xml
D:\Roll\DIP\SessionLogs\36\1.xml
D:\Roll\DIP\SessionLogs\36\2.xml
D:\Roll\DIP\SessionLogs\36\3.xml

I set inputs.conf: (in Universal forwarder)

[monitor://D:\Roll\DIP\SessionLogs\]
index = myindex
sourcetype = session_log

props.conf (in indexer)

[session_logs]
KV_MODE = xml

I dont get the logs in Search head ? Something am i missing here ..?

Re: Windows - How to monitor XML files within a sub-directory

Ayn — Wed, 27 Aug 2014 09:04:49 GMT

That looks OK. Make sure you're really searching for the logs correctly (specifying index for instance, searching over all time etc), and if you're sure the logs aren't really there, troubleshoot by checking splunkd.log on the forwarder. Also this script can be of help in order to determine the status of Splunk's file monitor: http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

Re: Windows - How to monitor XML files within a sub-directory

linu1988 — Wed, 27 Aug 2014 09:07:54 GMT

The below will monitor everything..

[monitor://D:\Roll\DIP\SessionLogs\...\*.xml]
index = myindex
sourcetype = session_log
recursive = true

Thanks,
L

Re: Windows - How to monitor XML files within a sub-directory

splunker12er — Wed, 27 Aug 2014 09:37:11 GMT

Note: A single dot (.) is not a wildcard, and is the regex equivalent of ..

Caution: In Windows, you cannot currently use a wildcard at the root level. For example, this does not work:

[monitor://E:\...\foo\*.log]
Splunk Enterprise logs an error and fails to index the desired files.

This is a known issue, described in the Known Issues topic of the Release Notes. Look there for details on all known issues.

Re: Windows - How to monitor XML files within a sub-directory

splunker12er — Wed, 27 Aug 2014 09:41:37 GMT

I tried in my windows universal forwarder the script , but cant execute it ,

C:\Program Files\SplunkUniversalForwarder\bin>splunk cmd python "c:\filestatus.py"
CreateProcess: The system cannot find the file specified.

couldn't run "c:\Program Files\SplunkUniversalForwarder\bin\python": The system cannot find the file specified.

Re: Windows - How to monitor XML files within a sub-directory

splunker12er — Wed, 27 Aug 2014 09:43:40 GMT

I tried ,

>splunk list monitor

Its shows the list of files & directories that are being monitored, but still cant view the data in SH. also there is no any errors in splunkd log.

Re: Windows - How to monitor XML files within a sub-directory

splunker12er — Thu, 28 Aug 2014 07:29:39 GMT

Below Works good :

At forwarder : (inputs.conf)

[monitor://D:\Roll\DIP\SessionLogs\]
recursive = true
index = myindex
sourcetype = session_log
whitelist = \.xml$

At Indexer: (props.conf)

[session_log]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
LINE_BREAKER = (</Data>)             ###Last element of the XML file
MAX_TIMESTAMP_LOOKAHEAD = 150
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = False
pulldown_type = 1