Getting Data In

Update existing index with CSV files containing changes

wredny125
New Member

Hi Guys,

I have a case where I'm importing every week a new dump of a data base to Splunk index using CSV files (I'm deleting the index with old data, and then recreating it back with new once). Right now I will receive not the whole dump of the DB in CSV but only CSV files with the items that have recently changed or have been added.

What would be the best way to update my existing index with this incremental csv files in Splunk and how could I do that?

Many thanks in advance

Tags (3)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Splunk doesn't do updates on index data, it's insert-only.

You could insert a new version of an event, and select the most recent version in your search, but that can be complicated and expensive.
You could store the entire set in a lookup file or Splunk's Key-Value-Store which both support updates but may or may not be suitable for your use case.
Have you considered leaving the data in an RDBMS and querying that from Splunk at search time using DB Connect?

Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...