topic Text logs not forwarded or indexed in Getting Data In

Text logs not forwarded or indexed

MattMorish — Thu, 20 Aug 2020 14:51:49 GMT

Hi,

I have an issue in forwarding application logs (text files) to splunk.
Windows Event Logs are forwarded and indexed properly but text files in a local drive are not.
Do you have any idea what the cause of this problem?

Thank you.

Re: Text logs not forwarded or indexed

isoutamo — Thu, 20 Aug 2020 15:09:16 GMT

can you send your inputs.conf and outputs.conf files so community can help you?

r. Ismo

Re: Text logs not forwarded or indexed

MattMorish — Thu, 20 Aug 2020 15:45:17 GMT

HI,

Thank you for your quick reply. I'm attaching the conf.

--- inputs.conf ---

[monitor://\\vmappt123\CREMS-PLUS-IF\LOG\]
disabled = false
index = main
sourcetype =
whitelist = *.*

[monitor://\\vmappt123\CREMS-PLUS-IF\LOG]
disabled = false
index = main

[monitor://\\vmappt123\CREMS-PLUS-IF\LOG\*.log*]
disabled = false
index = main

[monitor://D:\App\IVIF0001DJ\LOG]
disabled = false
index = main

[monitor://D:\App\IVIF0001DJ\LOG\*]
disabled = false
index = default
sourcetype = MXG-CREMS Plus - Realtime

[monitor://\\vmappp123\CREMS-PLUS-IF\LOG\*]
disabled = false
index = main
sourcetype = MXG-CREMS Plus - Realtime

[monitor://\\vmappt123\CREMS-PLUS-IF\LOG\*]
disabled = false
index = main

[monitor://\\vmappp123\CREMS-PLUS-IF\LOG]
disabled = false
index = main

[monitor://D:\App\IVIF0001DJ\LOG\*.log*]
disabled = false
index = main
sourcetype = MXG-CREMS Plus - Realtime

[monitor://\\vmappp123\CREMS-PLUS-IF\LOG\*.log*]
disabled = false
index = main

--- End of inputs.conf ---

--- outputs.conf ---

[tcpout]
defaultGroup = my_search_peers

[tcpout:my_search_peers]
server = v-spk01p:9997, v-spk01c:9997
autoLB = true

--- End of outputs.conf ---

Re: Text logs not forwarded or indexed

isoutamo — Thu, 20 Aug 2020 16:32:59 GMT

Anyone of those is working?

Are your splunk user in UF using local, domain or service user? Based on your inputs it should be a domain or service.

Which version you are using?

r. Ismo

Re: Text logs not forwarded or indexed

MattMorish — Thu, 20 Aug 2020 17:32:25 GMT

Hi,

None of them is working, but according to TailingProcessor:FileStatus, it looks that the forwarder is successfully reading all files.

"Local System account" is selected on the tab "Log On" in the property of "SplunkForwarder Service" running on the source server.

We are using Splunk Enterprise 6.6.6.

Thank you.

Re: Text logs not forwarded or indexed

isoutamo — Thu, 20 Aug 2020 18:42:29 GMT

If I recall right, remote files should read with domain account not local? You could try to reset any of filepointer in fishbucket and see if it reread it. Otherwise I propose to use domain account.

In whitelist those are regex so try .*\..* instead of *.*

r. Ismo

Re: Text logs not forwarded or indexed

MattMorish — Tue, 25 Aug 2020 14:01:25 GMT

Hi,

Thank you very much for your helpful suggestion! We cracked the issue by doing as below.
1) Removed all the forwarded inputs that referenced network shares and just used local directories. Only 3 are present now (one of which must be working):

2) Renamed one of the log files to have the extension .log

3) Adding a new line, the text has appeared in Splunk

We think its the third one we tried that's based on the regular expression that you suggested.
Thank you very much for pointing us the right direction!