Getting Data In
Highlighted

Windows - How to monitor XML files within a sub-directory

Motivator

I want to monitor XML files residing inside sub-directories.

Files inside Path :

D:\Roll\DIP\SessionLogs\35\1.xml
D:\Roll\DIP\SessionLogs\35\2.xml
D:\Roll\DIP\SessionLogs\35\3.xml
D:\Roll\DIP\SessionLogs\36\1.xml
D:\Roll\DIP\SessionLogs\36\2.xml
D:\Roll\DIP\SessionLogs\36\3.xml

I set inputs.conf: (in Universal forwarder)

[monitor://D:\Roll\DIP\SessionLogs\]
index = myindex
sourcetype = session_log

props.conf (in indexer)

[session_logs]
KV_MODE = xml

I dont get the logs in Search head ? Something am i missing here ..?

Highlighted

Re: Windows - How to monitor XML files within a sub-directory

Legend

That looks OK. Make sure you're really searching for the logs correctly (specifying index for instance, searching over all time etc), and if you're sure the logs aren't really there, troubleshoot by checking splunkd.log on the forwarder. Also this script can be of help in order to determine the status of Splunk's file monitor: http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

Highlighted

Re: Windows - How to monitor XML files within a sub-directory

Motivator

I tried in my windows universal forwarder the script , but cant execute it ,

C:\Program Files\SplunkUniversalForwarder\bin>splunk cmd python "c:\filestatus.py"
CreateProcess: The system cannot find the file specified.

couldn't run "c:\Program Files\SplunkUniversalForwarder\bin\python": The system cannot find the file specified.
0 Karma
Highlighted

Re: Windows - How to monitor XML files within a sub-directory

Motivator

I tried ,

>splunk list monitor

Its shows the list of files & directories that are being monitored, but still cant view the data in SH. also there is no any errors in splunkd log.

0 Karma
Highlighted

Re: Windows - How to monitor XML files within a sub-directory

Champion

The below will monitor everything..

[monitor://D:\Roll\DIP\SessionLogs\...\*.xml]
index = myindex
sourcetype = session_log
recursive = true

Thanks,
L

0 Karma
Highlighted

Re: Windows - How to monitor XML files within a sub-directory

Motivator

Note: A single dot (.) is not a wildcard, and is the regex equivalent of ..

Caution: In Windows, you cannot currently use a wildcard at the root level. For example, this does not work:

[monitor://E:\...\foo\*.log]
Splunk Enterprise logs an error and fails to index the desired files.

This is a known issue, described in the Known Issues topic of the Release Notes. Look there for details on all known issues.
0 Karma
Highlighted

Re: Windows - How to monitor XML files within a sub-directory

Motivator

Below Works good :

At forwarder : (inputs.conf)

[monitor://D:\Roll\DIP\SessionLogs\]
recursive = true
index = myindex
sourcetype = session_log
whitelist = \.xml$

At Indexer: (props.conf)

[session_log]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
LINE_BREAKER = (</Data>)             ###Last element of the XML file
MAX_TIMESTAMP_LOOKAHEAD = 150
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = False
pulldown_type = 1

View solution in original post