Reporting

populate_lookup creates a gziped (.gz) file?

Lowell
Super Champion

Anyone know why the populate_lookup saved search alert action creates a compressed .gz file even if the destination file doesn't end in ".gz"?

I have a savedsearches.conf entry like so:

[Metrics daily sourcetype - build lookup]
action.populate_lookup = 1
action.populate_lookup.dest = etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv
cron_schedule = 0 * * * *
dispatch.earliest_time = -27d@d
dispatch.latest_time = @d
displayview = flashtimeline
enableSched = 1
search = (earliest=-10d@d latest=@d date_hour::`this_hour`) OR (earliest=-27d@d latest=@d date_wday::`this_wday`) index=_internal tag::host=splunk source=*metrics.log "group=per_sourcetype_thruput" | .... and so on ...

When the savesearch runs, it actually generates a file named metrics_daily_sourcetype.csv.gz (note the ".gz" on the end). Is this normal behavior or is it a bug of some type?

I also see the following message in my splunkd log that corresponds to each time this saved search runs:

ERROR SearchOperator:copyresults - You must provide a search id.

And when I bumped up the logging on the copyresults operator, I got the following message too (but apparently only one time):

INFO  SearchOperator:copyresults - copy results.csv.gz to /opt/splunk/etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv.gz, success=1
1 Solution

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

View solution in original post

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

jeffa
Path Finder

I am seeing the same issue. Any update as to the cause?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...