Dashboards & Visualizations

Is it possible to use MySQL prepared statements in dbxquery?

TandyCowan
Engager

Splunk Enterprise 6.5.1 - yes, an upgrade is planned!

I have a series of equivalent dbxqueries in a Splunk dashboard panel, against two flavours of schema, whose basic form is:

<query>| dbxquery query="SELECT 'GAMMA_JETSONS' as CLIENT,
(SELECT GROUP_CONCAT(DISTINCT SOURCE order by SOURCE asc , ' , ' ) as source from gamma_jetsons.live_data) AS FEEDS_TODAY,
(SELECT COUNT(DISTINCT SOURCE) from gamma_jetsons.live_data) AS CURRENT,
(SELECT COUNT(DISTINCT SOURCE) from gamma_jetsons.live_data_archived) AS YESTERDAY,
(SELECT GROUP_CONCAT(DISTINCT SOURCE order by SOURCE asc , ' , ' ) as source from gamma_jetsons.live_data_archived) AS FEEDS_YESTERDAY
FROM DUAL;" 
connection="PROD_GAMMA_JETSONS" shortnames=1 | fields - _raw, _time</query>

In order to make it easier to change schema and add new rows for new databases I have tried to parameterise this using MySQL Prepared Statements, which (escaped for quotation marks and semicolons and less-than symbols) is:

<query>| dbxquery query="SET @my_cust:='GAMMA_JETSONS'&#59;
SET @my_custid:='jetsons'&#59;
SET @t1s:=CONCAT(&quot;SELECT IF((SELECT COUNT(SCHEMA_NAME) from information_schema.schemata WHERE SCHEMA_NAME='&quot;,@my_custid,&quot;_engine_persist_service') < 1,'gamma_&quot;,@my_custid,&quot;.live_data','&quot;,@my_custid,&quot;_engine_persist_service.live_data') INTO @t1;&quot;)&#59;
SET @t2s:=CONCAT(&quot;SELECT IF((SELECT COUNT(SCHEMA_NAME) from information_schema.schemata WHERE SCHEMA_NAME='&quot;,@my_custid,&quot;_engine_persist_service') < 1,'gamma_&quot;,@my_custid,&quot;.live_data_archived','&quot;,@my_custid,&quot;_engine_persist_service.live_data_archived') INTO @t2;&quot;)&#59;
PREPARE stmt FROM @t1s&#59; EXECUTE stmt&#59; DEALLOCATE PREPARE stmt&#59;
PREPARE stmt FROM @t2s&#59; EXECUTE stmt&#59; DEALLOCATE PREPARE stmt&#59;
SET @my_sql:= CONCAT(&quot;SELECT '&quot;,@my_cust,&quot;' as CLIENT,
(SELECT GROUP_CONCAT(DISTINCT source ORDER BY source ASC) AS source FROM &quot;,@t1,&quot;) AS FEEDS_TODAY,
(SELECT COUNT(DISTINCT SOURCE) FROM &quot;,@t1,&quot;) AS CURRENT,
(SELECT COUNT(DISTINCT SOURCE) FROM &quot;,@t2,&quot;) AS YESTERDAY,
(SELECT GROUP_CONCAT(DISTINCT source ORDER BY source ASC) AS source FROM &quot;,@t2,&quot;) AS FEEDS_YESTERDAY
FROM DUAL;&quot;)&#59;
PREPARE stmt FROM @my_sql&#59; EXECUTE stmt&#59; DEALLOCATE PREPARE stmt&#59;" 
connection="PROD_GAMMA_JETSONS" shortnames=1 | fields - _raw, _time</query>

The unescaped version works fine in MySQL using e.g. HeidiSQL:

SET @my_cust:='GAMMA_JETSONS';
SET @my_custid:='jetsons';
SET @t1s:=CONCAT("SELECT IF((SELECT COUNT(SCHEMA_NAME) from information_schema.schemata WHERE SCHEMA_NAME='",@my_custid,"_engine_persist_service') < 1,'gamma_",@my_custid,".live_data','",@my_custid,"_engine_persist_service.live_data') INTO @t1;");
SET @t2s:=CONCAT("SELECT IF((SELECT COUNT(SCHEMA_NAME) from information_schema.schemata WHERE SCHEMA_NAME='",@my_custid,"_engine_persist_service') < 1,'gamma_",@my_custid,".live_data_archived','",@my_custid,"_engine_persist_service.live_data_archived') INTO @t2;");
PREPARE stmt FROM @t1s; EXECUTE stmt; DEALLOCATE PREPARE stmt;
PREPARE stmt FROM @t2s; EXECUTE stmt; DEALLOCATE PREPARE stmt;
SET @my_sql:= CONCAT("SELECT '",@my_cust,"' as CLIENT,
(SELECT GROUP_CONCAT(DISTINCT source ORDER BY source ASC) AS source FROM ",@t1,") AS FEEDS_TODAY,
(SELECT COUNT(DISTINCT SOURCE) FROM ",@t1,") AS CURRENT,
(SELECT COUNT(DISTINCT SOURCE) FROM ",@t2,") AS YESTERDAY,
(SELECT GROUP_CONCAT(DISTINCT source ORDER BY source ASC) AS source FROM ",@t2,") AS FEEDS_YESTERDAY
FROM DUAL;");
PREPARE stmt FROM @my_sql; EXECUTE stmt; DEALLOCATE PREPARE stmt;

In Splunk all I get is an error in the panel's header row:

error_message=A value for dbxquery command option connection is required

I can't see what the issue might be. Am I missing something, or are Prepared Statements and multi-statement queries of this kind not supported?

Thanks.

Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...