All Apps and Add-ons

Implementing custom task handling across many users: KVStore vs external SQL-DB?

alvaromari83
Path Finder

Hello all!

I'm implementing a "task ticketing model" in my Splunk apps. Based on roles, many users may be assigned to a task, update it, and store new task statuses and other task fields.

I've been doing this with summary indexing (collect commands), but now that many users can generate new task, or operate over the same task concurrently, I'm having concurrency issues (duplicated ids, and other data integrity problems).

As far as I know, apps like Enterprise Security use KVStore to handle this kind of multiuser workflow scenarios, but I'm not sure if this is the most reliable, concurrency failure free, and capable of have relatively big sizes/records (aprox 100000 per year).
Performance is also an important factor.

My other option is to set up an external relational SQL DBMS and use it through DB Connect.

Any light on this to help me would be greatly appreciated!

Thank you!

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

http://www.mongodb-is-web-scale.com

You need locking of a structured record... a traditional database is the best tool IMHO.

0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...