Dashboards & Visualizations

[Studio JSON Source Code] Possible to append two ds.search or ds.chain together WITHOUT subsearches?

ClubMed
Path Finder

Specifically speaking the dataSources section discussed here:
https://docs.splunk.com/Documentation/Splunk/9.2.1/DashStudio/dashDef#The_dataSources_section

 

Hypothetically, I have two tables, each stored in individual data source stanzas:

  • Table 1 = ds.search stanza 1
  • Table 2 = ds.search stanza 2

The goal is to append the tables together, and then use the "stats join" method to merge the two tables together. If possible, this merge could be done as a ds.chain type stanza with two extend options, but it does not appear to be allowed.

ClubMed_1-1717042065345.png

Here's the documentation for Data source options.

https://docs.splunk.com/Documentation/Splunk/9.2.1/DashStudio/dsOpt 

The document seems to be missing options like "extend", so I'm hoping someone knows if there's any additional options that is hidden.

Now, I am trying to avoid using the [] subsearches because of 50,000 row limit, so the following append command will not be desired:

<base search> | append [search ....]

Anyone with mastery of JSON hacks might know if appending two data sources stanzas together be possible?

Thank you.

Labels (1)
0 Karma
1 Solution

ClubMed
Path Finder

After an investigation long story short, this is not possible and it needs to be a new feature suggestion if someone needs to request it.

The explanation is that I looked at the network logs for the Dashboard Studio and found the payload for base and chain searches.

Base search has their own parameter in the payload called 'search'.

All chained searches are grouped together in a parameter called 'postprocess_searches'.

There's no other parameters that support a third search parameter and call it 'append'. It is in fact non-existent with the payload structure.

Furthermore, based on the name 'postprocess_searches' parameter, it is clear only the base search gets distributable commands. All post-process searches (chained searches) happen on the searchhead only.

That is an important rule to keep in mind. If you want your search to be fast, then all the compute-heavy commands need to be in the base search.

Unfortunately, that means you'd need your base search to be a relatively large table of all sourcetypes appended together into a single table and do whatever necessary aggregation is required. Then use chained searches to slice and dice this large table into small bits, such as dividing by 'sourcetype' to branch out the table into multiple smaller "base" tables as the basis for additional chained searches.

In my case, I formulated my base search to be a merge of 3 different sourcetypes using stats join. It is reasonably fast with the ability to distribute the base search, despite having 15+ chained searches running off of it!

View solution in original post

0 Karma

ClubMed
Path Finder

After an investigation long story short, this is not possible and it needs to be a new feature suggestion if someone needs to request it.

The explanation is that I looked at the network logs for the Dashboard Studio and found the payload for base and chain searches.

Base search has their own parameter in the payload called 'search'.

All chained searches are grouped together in a parameter called 'postprocess_searches'.

There's no other parameters that support a third search parameter and call it 'append'. It is in fact non-existent with the payload structure.

Furthermore, based on the name 'postprocess_searches' parameter, it is clear only the base search gets distributable commands. All post-process searches (chained searches) happen on the searchhead only.

That is an important rule to keep in mind. If you want your search to be fast, then all the compute-heavy commands need to be in the base search.

Unfortunately, that means you'd need your base search to be a relatively large table of all sourcetypes appended together into a single table and do whatever necessary aggregation is required. Then use chained searches to slice and dice this large table into small bits, such as dividing by 'sourcetype' to branch out the table into multiple smaller "base" tables as the basis for additional chained searches.

In my case, I formulated my base search to be a merge of 3 different sourcetypes using stats join. It is reasonably fast with the ability to distribute the base search, despite having 15+ chained searches running off of it!

0 Karma

PickleRick
SplunkTrust
SplunkTrust

There's a portal for such feature requests - https://ideas.splunk.com/

 

0 Karma

ClubMed
Path Finder

Update in case anyone tried testing to see if "append" option exists, the "append" option does actually save but appears to not work.

ClubMed_0-1717042948144.png

 

0 Karma
Get Updates on the Splunk Community!

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...

Part 2: A Guide to Maximizing Splunk IT Service Intelligence

Welcome to the second segment of our guide. In Part 1, we covered the essentials of getting started with ITSI ...

Part 1: A Guide to Maximizing Splunk IT Service Intelligence

As modern IT environments continue to grow in complexity and speed, the ability to efficiently manage and ...