All Apps and Add-ons

DB Connect and SQLite database locking

responsys_cm
Builder

I have about 30 searches that run once per day and pull data from a SQLite database and write them to Splunk lookup tables. Most of the searches only take a few seconds to run. Each search is scheduled a minute apart. Yet it seems that I'm getting errors when most of the scripts run indicating the database is locked because it is busy:

2016-09-20 00:00:34.936 dbx8905:ERROR:DatabaseQueryCommand - SQL exception occurred: [SQLITE_BUSY] The database file is locked (database is locked) java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked) at org.sqlite.DB.newSQLException(DB.java:383) at org.sqlite.DB.newSQLException(DB.java:387) at org.sqlite.DB.execute(DB.java:339) at org.sqlite.Stmt.exec(Stmt.java:65) at org.sqlite.Stmt.executeQuery(Stmt.java:122) at com.splunk.dbx.sql.Database.execute(Database.java:236) at com.splunk.dbx.command.DatabaseQueryCommand.executeCommand(DatabaseQueryCommand.java:83) at com.splunk.runtime.SplunkCommand.invoke(SplunkCommand.java:237) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:92) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:30) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Is there any way to adjust the DBX timeout or force it to only use a single thread when accessing a SQLite database?

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Once upon a time I worked for a company that reengineered its policy engine to sit on SQLite... and that happened. I think the query_timeout value in inputs.conf might do it for you? http://docs.splunk.com/Documentation/DBX/2.3.0/DeployDBX/inputsspec

Another idea might be to change your searches to dbxquery statements that | collect into indexes, though doing that 30 times sounds tedious.

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...