All Apps and Add-ons

DB Connect and SQLite database locking

responsys_cm
Builder

I have about 30 searches that run once per day and pull data from a SQLite database and write them to Splunk lookup tables. Most of the searches only take a few seconds to run. Each search is scheduled a minute apart. Yet it seems that I'm getting errors when most of the scripts run indicating the database is locked because it is busy:

2016-09-20 00:00:34.936 dbx8905:ERROR:DatabaseQueryCommand - SQL exception occurred: [SQLITE_BUSY] The database file is locked (database is locked) java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked) at org.sqlite.DB.newSQLException(DB.java:383) at org.sqlite.DB.newSQLException(DB.java:387) at org.sqlite.DB.execute(DB.java:339) at org.sqlite.Stmt.exec(Stmt.java:65) at org.sqlite.Stmt.executeQuery(Stmt.java:122) at com.splunk.dbx.sql.Database.execute(Database.java:236) at com.splunk.dbx.command.DatabaseQueryCommand.executeCommand(DatabaseQueryCommand.java:83) at com.splunk.runtime.SplunkCommand.invoke(SplunkCommand.java:237) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:92) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:30) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Is there any way to adjust the DBX timeout or force it to only use a single thread when accessing a SQLite database?

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Once upon a time I worked for a company that reengineered its policy engine to sit on SQLite... and that happened. I think the query_timeout value in inputs.conf might do it for you? http://docs.splunk.com/Documentation/DBX/2.3.0/DeployDBX/inputsspec

Another idea might be to change your searches to dbxquery statements that | collect into indexes, though doing that 30 times sounds tedious.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!