topic Re: inputs.conf issue in Getting Data In

inputs.conf issue

balcv — Wed, 26 May 2021 04:07:28 GMT

I have a host that I am receiving logs into my heavy forwarder and that works fine.

I now have a new log source on the same host and the entry in my inputs.conf is not passing the data I need through.

[monitor:///mnt/nfs/host/Backup/DHCP/2021-05-*] disabled = 0

The wildcard is to cover a longish string of text that forms the file name. For example /mnt/nfs/host/Backup/DHCP/2021-05-22-192.168.64.88.log.0.ExtractedOption82Data

I am not getting the data from this log file no matter what variation or combination I try. Even if I specify a specific file name, the data is not appearing in the search.

I've tried using the full path and file in the monitor stanza, I've tried just the path in monitor and then the filename in whitelist=() but the same result. No Data yet I know the files exist and they contain data.

This is driving me crazy as I have done similar things previously with no issue. What am I missing??

Re: inputs.conf issue

venkatasri — Wed, 26 May 2021 05:08:30 GMT

Hi @balcv

Can you paste full inputs conf monitor?

You can verify splunkd logs at path - Linux /opt/splunk<forwarder>/var/log/splunk/ or _internal index for any errors related to read permission issues to file

execute the command on host where monitor is configured to find out /opt/splunk<forwarder>/bin - ./splunk list inputstatus - your file should be listed here.

---------------------------------------------------------------------------

An upvote would be appreciated if it helps!

Re: inputs.conf issue

balcv — Tue, 08 Jun 2021 23:32:04 GMT

Hi @venkatasri

I have reviewed the file /opt/splunkforwarder/var/log/splunk/splunkd.log and the following message appears:

06-09-2021 09:29:42.407 +1000 INFO TailingProcessor - Parsing configuration stanza: monitor:///mnt/nfs/Yul/Splunk_Backup/DHCP/2021-06-01-xxx.xxx.xxx.xxx.log.0.ExtractedOption82Data.

06-09-2021 09:29:42.413 +1000 INFO TailingProcessor - Adding watch on path: /mnt/nfs/Yul/Splunk_Backup/DHCP/2021-06-01-xxx.xxx.xxx.xxx.log.0.ExtractedOption82Data.

06-09-2021 09:29:42.590 +1000 ERROR TailReader - File will not be read, is too small to match seekptr checksum (file=/mnt/nfs/Yul/Splunk_Backup/DHCP/2021-06-01-xxx.xxx.xxx.xxx.log.0.ExtractedOption82Data). Last time we saw this initcrc, filename was different. You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info

I hope this helps shed some light on the issues.

Thanks
Leigh

Re: inputs.conf issue

venkatasri — Wed, 09 Jun 2021 00:32:02 GMT

Hi @balcv

crcSalt = <SOURCE>

Set the above setting exactly and do not modify, your new inputs should look like below. Restart UF. Detailed read available here look at the Disclaimer in this post - Solved: Windows DHCP log files "too small to match seekptr... - Splunk Community

[monitor:///mnt/nfs/host/Backup/DHCP/2021-05-*] disabled = 0 crcSalt = <SOURCE>

---------

An upvote would be appreciated if it helps!

Re: inputs.conf issue

balcv — Wed, 09 Jun 2021 00:50:32 GMT

Thanks @venkatasri . That seems to have fixed that problem in that the files are now appearing which is awesome. The problem now is that when the files appear on the search head, they only contain 6 rows rather than the several thousand rows per file that should be there.

I have read that it may be because the first 5 rows in each log file is the same and has "descriptive" text about the log file. Could this be a cause?

Thanks

Leigh

Re: inputs.conf issue

venkatasri — Wed, 09 Jun 2021 03:40:55 GMT

Hi @balcv

crcSalt setting shall work without issues, can you expand timerange in UI while searching it, reason could be your first 5 lines having no timestamp and events right after 5 line might be having timestamp which could have been appearing in other time-window.

Other option is to set initCrcLength , Try initCrcLength = 1500, hope your first 5 lines of file are below 1500 chars if not increase this. Read more here - inputs.conf - Splunk Documentation

------

An upvote would be appreciated if it helps!

Re: inputs.conf issue

balcv — Wed, 09 Jun 2021 04:38:46 GMT

Thanks @venkatasri . I think it's to do with the timestamp. If I expand the time period then records start appraising for some of the older files, however the more current files still only show a few records. I'm working on getting the leading text removed from the log files so that all I have is the log data. Hopefully that will resolve things.

Thanks for your help.

Cheers

Re: inputs.conf issue

venkatasri — Wed, 09 Jun 2021 04:41:55 GMT

Thanks @balcv for sharing the results. If your original problem was resolved 'Accept the solution' that would be appreciated.