All Apps and Add-ons

DB Connect and SQLite database locking

responsys_cm
Builder

I have about 30 searches that run once per day and pull data from a SQLite database and write them to Splunk lookup tables. Most of the searches only take a few seconds to run. Each search is scheduled a minute apart. Yet it seems that I'm getting errors when most of the scripts run indicating the database is locked because it is busy:

2016-09-20 00:00:34.936 dbx8905:ERROR:DatabaseQueryCommand - SQL exception occurred: [SQLITE_BUSY] The database file is locked (database is locked) java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked) at org.sqlite.DB.newSQLException(DB.java:383) at org.sqlite.DB.newSQLException(DB.java:387) at org.sqlite.DB.execute(DB.java:339) at org.sqlite.Stmt.exec(Stmt.java:65) at org.sqlite.Stmt.executeQuery(Stmt.java:122) at com.splunk.dbx.sql.Database.execute(Database.java:236) at com.splunk.dbx.command.DatabaseQueryCommand.executeCommand(DatabaseQueryCommand.java:83) at com.splunk.runtime.SplunkCommand.invoke(SplunkCommand.java:237) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:92) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:30) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Is there any way to adjust the DBX timeout or force it to only use a single thread when accessing a SQLite database?

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Once upon a time I worked for a company that reengineered its policy engine to sit on SQLite... and that happened. I think the query_timeout value in inputs.conf might do it for you? http://docs.splunk.com/Documentation/DBX/2.3.0/DeployDBX/inputsspec

Another idea might be to change your searches to dbxquery statements that | collect into indexes, though doing that 30 times sounds tedious.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...