Deployment Architecture

Is there a limit to the number of records that dboutput can update at a time?

NigelCooke
Explorer

I occasionally get this error when using the dboutput command :

command="dboutput", Unexpected error while performing dboutput: java.lang.ArrayIndexOutOfBoundsException: 100

It appears to relate to the number of records that the command is trying to update. Certainly, whenever the command succeeds the record count is less than 101. I have applied a "| head 100" command to ensure the record count is 100 or less. This seems to be successful in avoiding the error but I am now losing records.

If there is a limit it should be configurable should it not? In this case it looks like the source is hard coded for 100 records and is failing rather than a limit being applied.

Am I interpreting this error correctly?

1 Solution

Lucas_K
Motivator

Might be batch size.

http://docs.splunk.com/Documentation/DBX/latest/DeployDBX/javaspec

[dboutput]
batch.size = <n>

More info in this post : http://answers.splunk.com/comments/88886/view.html

Seems to suggest that type=update uses the batch setting as the total it will support where as type=insert will use the setting as many times as required.

View solution in original post

0 Karma

NigelCooke
Explorer

That stopped it. I added the block

[dboutput]
batch.size = 110

to the java.conf file and restarted Splunk. I ran some tests and the error now happens at 110 as set.

Thanks for your help.

0 Karma

Lucas_K
Motivator

Might be batch size.

http://docs.splunk.com/Documentation/DBX/latest/DeployDBX/javaspec

[dboutput]
batch.size = <n>

More info in this post : http://answers.splunk.com/comments/88886/view.html

Seems to suggest that type=update uses the batch setting as the total it will support where as type=insert will use the setting as many times as required.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...