Getting Data In

How to use rsync to move frozen data to another location?

mdtoro
Explorer

I was looking at rsync to move some frozen buckets to another location.  One concern,  if rsync picks up new frozen data still being written  and removes the source file before splunk finishes.

I could add find and mtime but this really slows down rsync.

Concerns?

 

Labels (3)
0 Karma

mdtoro
Explorer

I was thinking it is easier to not touch anything recently created.  Create a list of files at least 2 days old.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That could work, too.

---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Consider using lsof to test if the file is still open before moving it.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...