topic Re: How can i index the log file from a windows smtp service? in Getting Data In

How can i index the log file from a windows smtp service?

svendewindt — Mon, 28 Sep 2020 15:43:38 GMT

Hi,

After a lot of searching, trying and bashing my head, i will drop my problem here. I would like to index the logfiles of a windows smtp service. I thought this would be easy, however I can't get it to work.

Splunk is installed on a debian along with some other tools. So to index Windows files, I need to install the universal forwarder. I installed version 6, so the Splunk_TA_windows is installed too. During the installation I cannot browse to c:\windows\System32\logfiles. It's not there. I can browse to the folder in Windows explorer... Ok, no problem, I select a directory to monitor and i will change it in inputs.conf.

I understand that i should edit

C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local\inputs.conf. It does not matter what i try, if I add c:\windows\System32\logfiles, the folder is not getting indexed. Any other folder will be indexed.

What is so special about this folder?
I see that Splunk also indexes the eventlog (system, security and application) how can I remove these? During the installation i made sure nothing was selected. I only want the folder to be indexed.
How do I make sure that the indexed info is not stored in the main index. I would like an index per server. Or is that a bad idea. I seems easy to remove an index when a server no longer exits.

The logfiles are in IIS format. I read some articles that Splunk can handle this, but I would need to change the props file. Is that the file on the forwarder or on the indexer?

Thanks for any ideas,
Sven

Re: How can i index the log file from a windows smtp service?

gkanapathy — Mon, 27 Jan 2014 16:43:55 GMT

Did you install the Universal Forwarder to run as a user with access to the files/directory you need? You don't need to do anything with the props.conf file right now.

Re: How can i index the log file from a windows smtp service?

svendewindt — Mon, 27 Jan 2014 19:10:44 GMT

I installed the Universal Forwarder with domain admin credentials. I can browse to the folder in windows without issues...

Re: How can i index the log file from a windows smtp service?

mkinsley_splunk — Wed, 29 Jan 2014 23:30:58 GMT

are any errors showing up in your _internal index?

Re: How can i index the log file from a windows smtp service?

mkinsley_splunk — Wed, 29 Jan 2014 23:34:14 GMT

oh , i'm not sure if this is just a typo in your question, but from what I can see the logfiles directory is CamelCase :

System32\LogFiles\ , not System32\logfiles

Re: How can i index the log file from a windows smtp service?

svendewindt — Thu, 30 Jan 2014 10:26:07 GMT

I'm not that familiar with splunk. Where can i find the errors in the _internal index?

Re: How can i index the log file from a windows smtp service?

svendewindt — Thu, 30 Jan 2014 10:31:49 GMT

I would like to add, that when change the UniversalForwarder service to run under .\administrator then the files that i manually add to c:\windows\System32\logfiles are getting indexed. The log files, created by the smtp service however or not indexed. So that rules out any typo.

It seems to be related to ntfs rights. I checked the rights and they look correct. Just to be sure i forced the inheritance again. Still, the manually added files are indexed, the log files created by the smtp service are not indexed.

Re: How can i index the log file from a windows smtp service?

lukejadamec — Tue, 11 Feb 2014 19:00:36 GMT

The splunk logs are located in splunk\var\log\splunk\. The log file you're interested in is the splunkd.log.
If there is a problem with the indexing or reading of that log file, then you should be able to search the _internal logs for messages about the file. Example:
index=_internal "*LogFileName*"
For LogFileName just use the name of the smtp logfile, no need to put the whole path.

Re: How can i index the log file from a windows smtp service?

thesteve — Tue, 11 Feb 2014 22:14:06 GMT

It sounds like it might be a permissions issue to me.

As an administrator, you have access to that directory. As the local system account, you might not.

I would try creating an account with local permissions to that directory and running the UniversalForwarder service under that account.

As a quick test, you can have it run under your credentials or under the credentials of an administrator. If it works, all you need to do is add a service account.

Re: How can i index the log file from a windows smtp service?

gkanapathy — Tue, 11 Feb 2014 23:14:52 GMT

Occasionally Windows files will not update their timestamp, or will not be readable while the writing process still has it open. You can try setting

alwaysOpenFile = true

to the monitor stanza in inputs.conf and see if that helps.

Re: How can i index the log file from a windows smtp service?

svendewindt — Wed, 12 Feb 2014 08:23:51 GMT

Indeed. If I run the UniversalForwarder as an administrator, the files are getting indexed. I will create a service account.

Re: How can i index the log file from a windows smtp service?

svendewindt — Wed, 12 Feb 2014 08:25:59 GMT

This seems like an excellent tip. I can imagine, you should always set this for windows service logfiles.

Re: How can i index the log file from a windows smtp service?

bravon — Thu, 16 Apr 2015 15:27:21 GMT

Also, when troubleshooting permission issues - ProcMon from Sysinternals is gold.
https://technet.microsoft.com/en-us/library/bb896645.aspx

Filter it to only show "Result"=ACCESS DENIED and "Proccess Name" begins with: splunk

Re: How can i index the log file from a windows smtp service?

aalanisr26 — Thu, 16 Apr 2015 15:30:21 GMT

check the tailing status of that directory:

Open a browser to:
https://serverwithuniversalforwarder:8089/services/admin/inputstatus/TailingProcessor%3AFileStatus