Reporting

populate_lookup creates a gziped (.gz) file?

Lowell
Super Champion

Anyone know why the populate_lookup saved search alert action creates a compressed .gz file even if the destination file doesn't end in ".gz"?

I have a savedsearches.conf entry like so:

[Metrics daily sourcetype - build lookup]
action.populate_lookup = 1
action.populate_lookup.dest = etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv
cron_schedule = 0 * * * *
dispatch.earliest_time = -27d@d
dispatch.latest_time = @d
displayview = flashtimeline
enableSched = 1
search = (earliest=-10d@d latest=@d date_hour::`this_hour`) OR (earliest=-27d@d latest=@d date_wday::`this_wday`) index=_internal tag::host=splunk source=*metrics.log "group=per_sourcetype_thruput" | .... and so on ...

When the savesearch runs, it actually generates a file named metrics_daily_sourcetype.csv.gz (note the ".gz" on the end). Is this normal behavior or is it a bug of some type?

I also see the following message in my splunkd log that corresponds to each time this saved search runs:

ERROR SearchOperator:copyresults - You must provide a search id.

And when I bumped up the logging on the copyresults operator, I got the following message too (but apparently only one time):

INFO  SearchOperator:copyresults - copy results.csv.gz to /opt/splunk/etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv.gz, success=1
1 Solution

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

View solution in original post

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

jeffa
Path Finder

I am seeing the same issue. Any update as to the cause?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Introducing Splunk 10.0: Smarter, Faster, and More Powerful Than Ever

Now On Demand Whether you're managing complex deployments or looking to future-proof your data ...

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...