Reporting

populate_lookup creates a gziped (.gz) file?

Lowell
Super Champion

Anyone know why the populate_lookup saved search alert action creates a compressed .gz file even if the destination file doesn't end in ".gz"?

I have a savedsearches.conf entry like so:

[Metrics daily sourcetype - build lookup]
action.populate_lookup = 1
action.populate_lookup.dest = etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv
cron_schedule = 0 * * * *
dispatch.earliest_time = -27d@d
dispatch.latest_time = @d
displayview = flashtimeline
enableSched = 1
search = (earliest=-10d@d latest=@d date_hour::`this_hour`) OR (earliest=-27d@d latest=@d date_wday::`this_wday`) index=_internal tag::host=splunk source=*metrics.log "group=per_sourcetype_thruput" | .... and so on ...

When the savesearch runs, it actually generates a file named metrics_daily_sourcetype.csv.gz (note the ".gz" on the end). Is this normal behavior or is it a bug of some type?

I also see the following message in my splunkd log that corresponds to each time this saved search runs:

ERROR SearchOperator:copyresults - You must provide a search id.

And when I bumped up the logging on the copyresults operator, I got the following message too (but apparently only one time):

INFO  SearchOperator:copyresults - copy results.csv.gz to /opt/splunk/etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv.gz, success=1
1 Solution

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

View solution in original post

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

jeffa
Path Finder

I am seeing the same issue. Any update as to the cause?

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...