Deployment Architecture

Splunk replication of old data

rteja9
Path Finder

I have a 5 node Indexer cluster with version 7.3.1.1. I added configuration to replicate data on indexer cluster but only new data is getting replicated. Old data which is in cluster before I added replication configuration is not replicated.
Is this how it is supposed to be? If not, how can I replicate old data?

Thanks in advance!!

Labels (1)
Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @rteja9,
Splunk replicates only new data it isn't possible to replicate already indexed data, to replicate old data you have to reindex them.

If you don't want to reindex all the old data, the only solution is the one I used last year:

  • I renamed every index in index_new,
  • I created an eventtype for each index ( index=my_index OR index=my_index_new ),
  • I replaced in every search index=my_index with eventtype=my_eventtype,
  • I copied the old index in each indexer of my cluster (obviously restarting Splunk).

in this way you have a searchable copy of your data in every indexer, and the old data will slowly decrease until the old index will be empty after the retention period.

Ciao.
Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @rteja9,
Splunk replicates only new data it isn't possible to replicate already indexed data, to replicate old data you have to reindex them.

If you don't want to reindex all the old data, the only solution is the one I used last year:

  • I renamed every index in index_new,
  • I created an eventtype for each index ( index=my_index OR index=my_index_new ),
  • I replaced in every search index=my_index with eventtype=my_eventtype,
  • I copied the old index in each indexer of my cluster (obviously restarting Splunk).

in this way you have a searchable copy of your data in every indexer, and the old data will slowly decrease until the old index will be empty after the retention period.

Ciao.
Giuseppe

rteja9
Path Finder

Thanks for your response. That makes sense.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...