Hi all, I am attempting to create an add-on with the Add-On Builder that queries a REST API for stats and saves them to a metrics index. A requirement is that it needs to be smart during outages & recovery. I thought I could just add checkpointing to the add-on. As a test, I stopped Splunk for a few minutes, then started it up and got this: The spike is where Splunk was restarted. What appears to be happening is that the checkpoint is starting where it left off but just doing 1 query from where it left off to the current time and returning the sum. What I want it to do is basically a backfill - start at the saved checkpoint & run the REST query every 30 seconds and save those values. Is that possible to do in the Add-On builder or does that require additional coding? ... View more

Hello! I have been trying to get some logs into a metric index and I'm wondering if they can be improved with better field extraction. These are what the logs look like:     t=1713291900 path="/data/p1/p2" stat=s1:s2:s3:s4 type=COUNTER value=12 t=1713291900 path="/data/p1/p2" stat=s1:s2:s5:s6 type=COUNTER value=18 t=1713291900 path="/data/p1/p2" stat=s1:s2:s3:s7 type=COUNTER value=2 t=1713291900 path="/data/p1/p2" stat=s1:s2:s3 type=COUNTER value=104 t=1713291900 path="/data/p1/p2" stat=s1:s2:s3 type=COUNTER value=18 t=1713291900 path="/data/p1/p2" stat=s1:s2:s5:s8 type=COUNTER value=18 t=1713291900 path="/data/p1/p2" stat=s1:s2:s5:s8:s9:10 type=COUNTER value=8 t=1713291900 path="/data/p1/p2" stat=s1:s2:s3:s4 type=COUNTER value=104 t=1713291900 path="/data/p1/p2" stat=s1:s2:s5:s8:s9 type=COUNTER value=140 t=1713291900 path="/data/p1/p2" stat=s1:s2:s5:s8:s9 type=COUNTER value=3 t=1713291900 path="/data/p1/p2" stat=s1:s2:s5:s8:s9 type=COUNTER value=1 t=1713291900 path="/data/p3/p4" stat=s20 type=COUNTER value=585 t=1713291900 path="/data/p3/p4" stat=s21 type=COUNTER value=585 t=1713291900 path="/data/p3/p4" stat=s22 type=TIMEELAPSED value=5497.12 t=1713291900 path="/data/p3/p5" stat=s23 type=COUNTER value=585 t=1713291900 path="/data/p1/p5" stat=s24 type=COUNTER value=585 t=1713291900 path="/data/p1/p5" stat=s25 type=TIMEELAPSED value=5497.12 t=1713291900 path="/data/p1/p5/p6" stat=s26 type=COUNTER value=253 t=1713291900 path="/data/p1/p5/p6" stat=s27 type=GAUGE value=1     t is the epoch time. path is the path of a URL which is in double quotes, always starts with /data/, and can have anywhere between 2 and 7 (maybe more) subpaths. stat is is either a single stat (like s20) OR a colon-delimited string of between 3 and 6 stat names. type is either COUNTER, TIMEELAPSED, or GAUGE. value is the metric. Right now I've been able to get a metric index set up that: Assigns t as the timestamp and ignores t as a dimension or metric Makes value the metric Makes path, stat, and type dimensions This is my transforms.conf:     [metrics_field_extraction] REGEX = ([a-zA-Z0-9_\.]+)=\"?([a-zA-Z0-9_\.\/:-]+) [metric-schema:cm_log2metrics_keyvalue] METRIC-SCHEMA-MEASURES = value METRIC-SCHEMA-WHITELIST-DIMS = stat,path,type METRIC-SCHEMA-BLACKLIST-DIMS = t     And props.conf (it's basically log2metrics_keyvalue, we need cm_ to match to our license):     [cm_log2metrics_keyvalue] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) METRIC-SCHEMA-TRANSFORMS = metric-schema:cm_log2metrics_keyvalue TRANSFORMS-EXTRACT = metrics_field_extraction NO_BINARY_CHECK = true category = Log to Metrics description = '<key>=<value>' formatted data. Log-to-metrics processing converts the keys with numeric values into metric data points. disabled = false pulldown_type = 1      path and stat are extracted exactly as they appear in the logs. However, I'm wondering if it's possible to get each part in the path & stat fields into their own dimension, so instead of: _time path stat value type 4/22/24 2:20:00.000 PM /p1/p2/p3 s1:s2:s3 500 COUNTER   It would be: _time path1 path2 path3 stat1 stat2 stat3 value type 4/22/24 2:20:00.000 PM p1 p2 p3 s1 s2 s3 500 COUNTER   My thinking was that we'd be able to get really granular stats and interesting graphs. Thanks in advance! ... View more

Hello, I'm trying to add a simple date picker to a dashboard. I'm still pretty new to Splunk and Javascript so I'm hoping I'm just missing something simple. I've found other posts like: https://community.splunk.com/t5/Dashboards-Visualizations/using-splunk-UI-to-select-a-specific-date/m-p/481151 https://community.splunk.com/t5/Dashboards-Visualizations/Calendar-date-picker/m-p/49711 https://community.splunk.com/t5/Dashboards-Visualizations/Jquery-datepicker-in-splunk/m-p/361049 However I'm still unable to get what I'd like, which is 1 calendar so the user can pick a date. Most of the solutions are to select 2 dates for a range. For me, the range will always be 1 day as the dashboard only shows daily reports.  I did find https://medium.com/@nikolayryabykh/splunk-s01e03-the-one-with-js-and-css-b8c78851481c that has an example with a single date calendar picker but I'm having trouble making it work, maybe because it's from 2018. TL;DR, the instructions say to add:         <input type="text" token="token_date" id="date"> <label>Test textbox:</label> </input>         Then create a Javascript file with this:         require(["jquery", "splunkjs/mvc/simplexml/ready!"], function($) { $("[id^=date]") .attr('type', 'date') .attr('min', '2018-01-11') });       And add this:       <dashboard script="input.js">       I did Step 1 and 2 (via Upload Asset). I didn't do Step 3 as another procedure I was testing didn't require me to put script="input.js" in the form tag. Thanks in advance!   ... View more

Hello, I've starting working with data models and Pivot. They've been working for the most part, but recently, Pivot has been throwing errors for a few of the data models like this when I click on Pivot from the Data Model editor: The search job has failed due to an error. You may be able view the job in the Job Inspector.     Error in 'TsidxStats': Could not find job directory for sid: 1681752470.124849_22F608DA-1DAB-4C64-BCA3-3D6EA29D58A4   The sid changes each time I run Pivot. I've found the tstats command in search.log, and if I run that as its own search outside of Pivot, sometimes it runs fine, sometimes it doesn't.   This seems to have corresponded with a recent change changing our events to JSON format. Could that be the issue? No reports, dashboards, or alerts seem to have been affected by that change.    Thank you in advance! ... View more

Hello, I've been asked to create a report that will show the number of events from the 2 previous quarters by country, the monthly average, and the quarterly percent increase: Country Q1'22 Total Q1'22 Monthly Avg Q2'22 Total Q2'22 Monthly Avg Q2'22 Percent Increase US 300000 100000 330000 110000 10% UK 60000 20000 61000 20333 2% Canada 1200 400 1500 500 25%   Using this:     index=mydata earliest=-2q@q latest=-q@q | chart dc(ID) as count_earlier by Country | appendcols [ search index=mydata earliest=-q@q latest=@q | chart dc(ID) as count_later by Country] | eval ave_earlier=round(count_earlier/3,0) | eval ave_later=round(count_later/3,0) | eval DiffPer=round(((count_later - count_earlier) / count_earlier) * 100,0)."%" | table ReportersCountry,count_earlier,ave_earlier,count_later,ave_later,DiffPer     Now I'm trying to rename count_earlier, ave_earlier, count_later, and ave_later to be the quarter labels. I've been using:     | convert TIMEFORMAT="%m" ctime(_time) AS month | rex field=date_year "\d{2}(?<short_year>\d{2})" | eval quarter=case(month<=3,"Q1",month<=6,"Q2",month<=9,"Q3",month<=12,"Q4",1=1,"missing")."'".short_year     And have been trying to use eval {} to rename the columns but haven't quite figured it out. I also tried using chart which allows me to get the quarter headers, but then I couldn't figure out how to calculate the percent difference column.  Thanks for any help in advance! ... View more