Installation

Issues with data ingestion from the same xml files with appended data/raw

SplunkDash
Motivator

Hello,

I have xml source files in a location where SPLUNK HF installed on it...these files are updated with new data...that means no new files are created .....when new data comes in... existing files are appended, and new data added at the end of the existing files. I know SPLUNK has CRC to check duplicate entries based on the Hash Function. My question is.... will SPLUNK UF/HF be able to read only new data that is added at the end of existing file without any errors (or missing info) and send to the indexer. 

Any recommendations/thoughts would be highly appreciated. Thank you so much for your support.

Labels (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Normally monitor inputs - as the name says - monitor files for changes. If there is data appended to the end of the file, it _should_ be read and ingested. There are some border cases where it fails, often involving sharing files over the network, but in general case - it should work pretty well.

Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...