We have an on-prem Splunk instance (was 7.0.3, have now upgraded to 8.0.4 but are still seeing the same behaviour).
When I try to index files, or DBX connections, the file indexer correctly report...
See more...
We have an on-prem Splunk instance (was 7.0.3, have now upgraded to 8.0.4 but are still seeing the same behaviour).
When I try to index files, or DBX connections, the file indexer correctly reports the number of matching files in the directory, but the Index shows 0 events.
I have also tried indexing databases using DBConnect, which again, shows results during initial testing and configuration, but after setup, the index remains with 0 events in it. The $SPLUNK_HOME/var/log/splunk/splunk_app_db_connect_server.log file shows this:
2020-04-24 04:15:27.525 +0000 [QuartzScheduler_Worker-25] INFO org.easybatch.core.job.BatchJob - Job '<JOBNAME>' starting
2020-04-24 04:15:27.525 +0000 [QuartzScheduler_Worker-25] INFO org.easybatch.core.job.BatchJob - Batch size: 1,000
2020-04-24 04:15:27.525 +0000 [QuartzScheduler_Worker-25] INFO org.easybatch.core.job.BatchJob - Error threshold: N/A
2020-04-24 04:15:27.525 +0000 [QuartzScheduler_Worker-25] INFO org.easybatch.core.job.BatchJob - Jmx monitoring: false
2020-04-24 04:15:27.626 +0000 [QuartzScheduler_Worker-25] INFO c.s.d.s.dbinput.recordreader.DbInputRecordReader - action=db_input_record_reader_is_opened task=<JOBNAME> query=SELECT * FROM "<DATABASE>"."dbo"."<TABLE>"
2020-04-24 04:15:27.726 +0000 [QuartzScheduler_Worker-25] INFO org.easybatch.core.job.BatchJob - Job '<JOBNAME>' started
2020-04-24 04:15:27.776 +0000 [QuartzScheduler_Worker-25] INFO c.s.dbx.server.dbinput.recordwriter.HecEventWriter - action=write_records batch_size=50
2020-04-24 04:15:27.776 +0000 [QuartzScheduler_Worker-25] INFO c.s.d.s.dbinput.recordwriter.HttpEventCollector - action=writing_events_via_http_event_collector
2020-04-24 04:15:27.776 +0000 [QuartzScheduler_Worker-25] INFO c.s.d.s.dbinput.recordwriter.HttpEventCollector - action=writing_events_via_http_event_collector record_count=50
2020-04-24 04:15:27.778 +0000 [QuartzScheduler_Worker-25] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
******<snip>*******
2020-04-24 04:15:27.778 +0000 [QuartzScheduler_Worker-25] ERROR org.easybatch.core.job.BatchJob - Unable to write records
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
******<snip>*******
2020-04-24 04:15:27.778 +0000 [QuartzScheduler_Worker-25] INFO org.easybatch.core.job.BatchJob - Job '<JOBNAME>' finished with status: FAILED
I've turned off SSL checkboxes, so assume it's a mismatch on port expectations based on some other googling, and can confirm that a SPL of:
| dbxquery query="SELECT TOP 10 * FROM \"<DATABASE>\".\"dbo\".\"<TABLE>\"" connection="<CONNECTION>"
returns results, just like the DB Connect configuration does.
I'm really struggling to discover any reason why my indexes aren't being populated, and would really appreciate any help.
P