Hello, I have an inputs.conf on my forwarder setup like this,
[monitor:///opt/jira-maestro/plugins/bintray_url/csv/*.csv]
index=bintray
sourcetype=csv
[monitor:///opt/jira-maestro/plugins/nessus/csv/*.csv]
index=nessus
sourcetype=csv
forwarder sends data for the 1st index, "bintray", but I cant get it to send for 2nd index "nessus"
I enabled DEBUG for Tailing Processor, getting tons of msg like this in splunkd.log
38915394548.tmp', does not match path='/opt/jira-maestro/plugins/nessus/csv' :Not a directory :Not a symlink
04-06-2017 16:07:04.657 -0400 DEBUG TailingProcessor - Skipping itemPath='/opt/atlassian/jira/temp/imageio2771437074019475859.tmp', does not match path='/opt/jira-maestro/plugins/nessus/csv' :Not a directory :Not a symlink
04-06-2017 16:07:04.663 -0400 DEBUG TailingProcessor - Skipping itemPath='/opt/atlassian/jira/temp/imageio1428026418037972330.tmp', does not match path='/opt/jira-maestro/plugins/nessus/csv' :Not a directory :Not a symlink
Not sure where else to troubleshoot. Spent entire day trying to get it to send data over.
Do you have any errors about file access permisisons or similar? Can the Splunk user read the files in the directory?
Try running:
splunk btool inputs list --debug
If that shows the information you expect just double check that the monitor information was printed on the startup of the forwarder.
Finally you might want to check the metrics log file and see if the log is mentioned (it might or might not mention the sourcetype/index/source depending on how busy the forwarder is), if it does then you might have an issue finding the data rather than an issue with the data getting indexed.
Good luck
Hello, I tried running,
splunk btool inputs list --debug
It shows correct syntax,
/opt/splunkforwarder/etc/apps/jira-maestro/local/inputs.conf [monitor:///opt/jira-maestro/plugins/nessus/csv/report.csv]
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/apps/jira-maestro/local/inputs.conf disabled = false
/opt/splunkforwarder/etc/system/default/inputs.conf host = $decideOnStartup
/opt/splunkforwarder/etc/apps/jira-maestro/local/inputs.conf index = nessus
/opt/splunkforwarder/etc/apps/jira-maestro/local/inputs.conf sourcetype = csv
Also tried copying the csv file to some other location ,for example /opt/test
/opt/test/report.csv
created a new input.conf,
[default]
index = nessus
[monitor:///opt/test]
whitelist = ^.*.csv
sourcetype = csv
disabled = false
initCrcLength = 1048575
crcSalt = /opt/test
Restarted forwarded, nothing gets sent to indexer, also tried modfying report.csv file to generate a change, using vim
04-07-2017 14:03:49.845 -0400 INFO WatchedFile - Will begin reading at offset=45491 for file='/opt/test/report.csv'.
04-07-2017 14:03:49.849 -0400 INFO WatchedFile - Resetting fd to re-extract header.
04-07-2017 13:53:06.534 -0400 WARN FileClassifierManager - The file '/opt/test/.report.csv.swp' is invalid. Reason: binary
04-07-2017 13:53:06.534 -0400 INFO TailReader - Ignoring file '/opt/test/.report.csv.swp' due to: binary
04-07-2017 13:53:10.667 -0400 WARN FileClassifierManager - The file '/opt/test/.report.csv.swp' is invalid. Reason: binary
04-07-2017 13:53:10.667 -0400 INFO TailReader - Ignoring file '/opt/test/.report.csv.swp' due to: binary
04-07-2017 13:53:13.984 -0400 INFO WatchedFile - Will begin reading at offset=45491 for file='/opt/test/report.csv'.
04-07-2017 13:53:13.984 -0400 INFO WatchedFile - Resetting fd to re-extract header.
04-07-2017 13:53:13.985 -0400 WARN TailReader - Insufficient permissions to read file='/opt/test/.report.csv.swp' (hint: No such file or directory , UID: 0, GID: 0).
04-07-2017 13:53:16.989 -0400 INFO WatchedFile - Resetting fd to re-extract header.
also tried injecting a new column into csv to keep track of timestamp in format of "2017-04-07 11:38:53,008"
Nothing is being sent to indexer. Indexer splunkd log doesnt show anything coming in for this report.csv All permissions are splunk user + 644 on the report.csv file