Knowledge Management

How to append on existing gz lookup file without temporary file

shanmu
New Member

Hi Team,

I have many(10+) large csv lookup files (200MB) being referred in multiple places. To improve optimisation, moved them into compressed files (*.csv.gz) and made changes in existing files.

In some cases, we need to retain the existing file data. Thus said,

| outputlookup *.csv.gz  ```will fail to retain old data```

Hence Planning to do as follows,

| inputlookup "old_file.csv.gz" | outputlookup "temp_file.csv" ```taking a backup ```
| Run a new search 
| outputlookup append=t "temp_file.csv" ```append  new search results to the temp file```
| inputlookup "temp_file.csv" | outputlookup "old_file.csv.gz"
| makeresults | where false() | outputlookup create_empty=true "temp_file.csv"

 
And this needs to be done on multiple places :(.

Is there any better way to perform this without creating/clearing temp files?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You cannot append to a gzipped file. As simple as that.

0 Karma
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...