i think it really depends on your use case. typically i would prefer to use the native splunk scheduling, but 150+ near identical reports reports being run on the first of each month, interfering with interactive splunk users, and occasionally exhausting cluster resources was not ideal.  im using a scheduled ci-cd process that executes this job in a docker container. ... View more

i am using a local copy of the dashboard because thats what suited my use case.  my scenario was that i had users creating 150+ identical monthly reports (eg copy/pasting a dashboard definition and changing a client identifier) and then scheduling those dashboard to run in the first 6 hours of the month. the pattern i am using currently will pull a list of client id's from a lookup, iterate over each client id (replacing the client id in the dashboard definition (as you don't appear to be able to provide inputs when asking splunk to generate the dashboard)), and generating a pdf.  this process also allows me to process these reports sequentially rather than scheduling these using the splunk scheduling interface (which is not particularly easy to schedule large amounts of reporting with). This reduces a significant amount of load in the environment at the beginning of each month. for those of you looking for a more complete example than what i have provided earlier, i've been using something very similar to this:  splunkcloud scripted dashboard generation (github.com) hope this helps. ... View more

this thread was very helpful in figuring out how to do this, however i did have some problems with the gt (>) and lt(<) operators in my searches.  to work around this i wrapped them in CDATA tags:  status<![CDATA[&gt;]]>199 AND status<![CDATA[&lt;]]>300   if anyone else is interested, this is my "final" script:  import requests import json import os from logzero import setup_logger logger = setup_logger(name="logger", level=20) splunk_endpoint = os.environ["SPLUNK_ENDPOINT"] splunk_username = os.environ["SPLUNK_USERNAME"] splunk_password = os.environ["SPLUNK_PASSWORD"] with open("dashboard.xml", encoding="utf-8") as xml_file: xml_dashboard = xml_file.read() with open("client_list.json", encoding="utf-8") as json_file: client_list = json.loads(json_file.read()) for client in client_list: logger.info(f"generating dashboard pdf for {client['name']} [{client['id']}]") # replace tokens in dashboard this_xml_dashboard = xml_dashboard.replace("--CLIENT--NAME--", client["name"]) this_xml_dashboard = xml_dashboard.replace("--CLIENT--ID--", client["id"]) # send xml dashboard to render endpoint render_url = f"{splunk_endpoint}/services/pdfgen/render" render_response = requests.post( render_url, auth=(splunk_username, splunk_password), params={ "input-dashboard-xml": this_xml_dashboard.encode(), "paper-size": "a4-landscape", } ) logger.info(f"render_response: {render_response.status_code}") # render endpoint returns a pdf if render_response.status_code == 200: outfile = (f"./output/dashboard-{client['name'].replace(' ', '_').lower()}.pdf") with open(outfile, "wb") as pdffile: pdffile.write(render_response.content) logger.info(f"wrote {filename} [{render_response.headers['Content-Length']} bytes]")   hope this helps. ... View more

This post has been super helpful, and in the interests of making this easier for others, i've created some examples for use with the VS Code HTTP rest client. https://gist.github.com/mark-vandenbos/c550d3f29f6e5991d4c13a5c75c8e358 thanks! ... View more

I've been trying to figure out a nice way to do this, and this is what what i have come up with:  https://gist.github.com/6d61726b760a/e6fef9c2e5d47c43a1ba2ae5bd659638 its a little python script runs a search on your deployment server which generates a list of guids, and then submits a delete request for each guid returned.  My search is simply looking for clients that haven't reported in more than 24h but you could come up with whatever query you like. ... View more

thanks @Sukisen1981, i expanded a little bit and came up with the following which gives me exactly what i need. index=pcf foundation=$foundation_token$ sourcetype=cf:valuemetric job_index="*" name=Capacity* | stats list(name),list(value) by _time,job_index | rename list(name) as name,list(value) as value | eval x=mvzip(name,value,":") | fields job_index,x | nomv x | rex field=x "CapacityTotalDisk:(?<CapacityTotalDisk>.*)\s+" | rex field=x "CapacityRemainingDisk:(?<CapacityRemainingDisk>.*)\s+" | rex field=x "CapacityTotalMemory:(?<CapacityTotalMemory>.*?)\s+" | rex field=x "CapacityAllocatedMemory:(?<CapacityAllocatedMemory>.*?)\s+" | rex field=x "CapacityRemainingMemory:(?<CapacityRemainingMemory>.*?)\s+" | rex field=x "CapacityTotalContainers:(?<CapacityTotalContainers>.*?)\s+" | rex field=x "CapacityRemainingContainers:(?<CapacityRemainingContainers>.*?)\s+" | eval CapacityAllocatedContainers=(CapacityTotalContainers-CapacityRemainingContainers) | fields _time,job_index,CapacityTotalMemory,CapacityAllocatedMemory,CapacityRemainingMemory,CapacityTotalDisk,CapacityRemainingDisk,CapacityTotalContainers,CapacityAllocatedContainers,CapacityRemainingContainers | stats avg(CapacityTotalMemory) as totalMemory, avg(CapacityAllocatedMemory) as allocatedMemory, avg(CapacityRemainingMemory) as remainingMemory, sparkline(avg(CapacityRemainingMemory)) as "remainingMemory trend", avg(CapacityTotalDisk) as totalDisk, avg(CapacityRemainingDisk) as remainingDisk, sparkline(avg(CapacityRemainingDisk)) as "remainingDisk trend", avg(CapacityTotalContainers) as totalContainers, avg(CapacityAllocatedContainers) as allocatedContainers, avg(CapacityRemainingContainers) as remainingContainers, sparkline(avg(CapacityRemainingContainers)) as "remainingContainers trend" by job_index | eval allocatedMemory=round(allocatedMemory,2), remainingMemory=round(remainingMemory,2), remainingDisk=round(remainingDisk,2), remainingContainers=round(remainingContainers,0), allocatedContainers=round(allocatedContainers,0) | sort +remainingMemory ... View more