All Apps and Add-ons

DB Connect and SQLite database locking

responsys_cm
Builder

I have about 30 searches that run once per day and pull data from a SQLite database and write them to Splunk lookup tables. Most of the searches only take a few seconds to run. Each search is scheduled a minute apart. Yet it seems that I'm getting errors when most of the scripts run indicating the database is locked because it is busy:

2016-09-20 00:00:34.936 dbx8905:ERROR:DatabaseQueryCommand - SQL exception occurred: [SQLITE_BUSY] The database file is locked (database is locked) java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked) at org.sqlite.DB.newSQLException(DB.java:383) at org.sqlite.DB.newSQLException(DB.java:387) at org.sqlite.DB.execute(DB.java:339) at org.sqlite.Stmt.exec(Stmt.java:65) at org.sqlite.Stmt.executeQuery(Stmt.java:122) at com.splunk.dbx.sql.Database.execute(Database.java:236) at com.splunk.dbx.command.DatabaseQueryCommand.executeCommand(DatabaseQueryCommand.java:83) at com.splunk.runtime.SplunkCommand.invoke(SplunkCommand.java:237) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:92) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:30) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Is there any way to adjust the DBX timeout or force it to only use a single thread when accessing a SQLite database?

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Once upon a time I worked for a company that reengineered its policy engine to sit on SQLite... and that happened. I think the query_timeout value in inputs.conf might do it for you? http://docs.splunk.com/Documentation/DBX/2.3.0/DeployDBX/inputsspec

Another idea might be to change your searches to dbxquery statements that | collect into indexes, though doing that 30 times sounds tedious.

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...