ITSI Glass Tables - JSON Extraction

[genesiusj]

Hi,

I added this to the D&V and not ITSI because the json used in glass table and dashboard studio should follow the same structure. Correct?

Here is a sample of the dataSources from my gt/db.

    "dataSources": {
        "ds_fRdCg1iz": {
            "type": "ds.search",
            "name": "STAR Application - ServiceHealthScore",
            "options": {
                "query": "`get_full_itsi_summary_kpi(SHKPI-f4eb7372-4e0a-4dcb-9e6d-056c3151feda)` `service_level_kpi_only` | timechart cont=false latest(alert_value) AS alert_value, latest(alert_color) AS alert_color"
            },
            "meta": {
                "kpiID": "SHKPI-f4eb7372-4e0a-4dcb-9e6d-056c3151feda",
                "serviceID": "f4eb7372-4e0a-4dcb-9e6d-056c3151feda"
            }
        },

Unlike the other values which include a key (type: ds_search OR name: STAR blah blah) the ds search name does not have a key. It is just ds_fRdCg1iz.

I need to create a table these values.

ds_fRdCg1iz, STAR Application - ServiceHealthScore

There are hundreds of these searches.

Thanks and God bless,
Genesius

[PickleRick]

I'm a bit confused. Where do you want to create this table? And how are you expecting to get that data?

Are you doing

| rest /servicesNS/a/b/data/ui/views/whatever

to get the source for a dashboard and want to parse it?

[genesiusj]

@PickleRick 

I've used spath as follows.

| rest splunk_server=local /servicesNS/nobody/SA-ITOA/itoa_interface/glass_table report_as=text fields="_key,identifying_name,description,acl,definition" ```1```
| eval value=spath(value,"{}") 
| mvexpand value ```14 glass tables```
| eval glass_table_name = spath(value, "identifying_name") 
| search glass_table_name = star ```1 glass table```
| eval description = spath(value, "description") ```this is blank ???```
| eval key = spath(value, "_key") ```is this the glass table ky: 2913043e-76e1-11f0-abf2-0050568f9bbf```
| eval acl = spath(value, "acl") ```what is acl? an access control list listing what permissions are used on the glass table?```
| rex field=acl "\"owner\"\:\s\"(?<owner>\w+)\"" ```glass table owner/creator```
| eval definition = spath(value, "definition") ```what is definition?```
```| fields - value```
| eval ds_json = spath(definition, "dataSources") ```breakout data sources```

 These spath commands worked until the last my last eval command.

Thanks in advance for your assistance.

God bless,
Genesius

[PickleRick]

Unfortunately I don't have a ITSI instance at hand and in ordinary DS dashboard the json seems to be located in a bit different place than in ITSI GT

[victor_menezes]

Hey @genesiusj ,

Yeah, those ids are not as good as names for tracking when it comes to reading by source, which is mostly my case. What I do is whenever building dashboards I always change the DS generated ID and its references downstream to named instead, like ds_panela or ds_panelb, so I know by looking where they are coming from.

But in your case, as you are already with hundreds created such action is way too much to think of, so let's try this:

| rest /servicesNS/-/-/data/ui/views/ splunk_server=local
| search title="<your dashboard title>"
| fields title eai:acl.app eai:data
| rex field="eai:data" max_match=0 "(?s)\"(?<ds_key>ds_[^\"]+)\"\\s*:\\s*\\{.*?\"name\"\\s*:\\s*\"(?<ds_name>[^\"]+)\""
| eval pair=mvzip(ds_key, ds_name, "||")
| mvexpand pair
| eval key=mvindex(split(pair,"||"),0),
name=mvindex(split(pair,"||"),1)
| table key name
| sort key

This should give you what you need, the key (id) and the "name" of each data source:

[genesiusj]

@victor_menezes 

Thanks. I'm trying to figure how to use that in my use case.

This is the code for my glass table.

| rest splunk_server=local /servicesNS/nobody/SA-ITOA/itoa_interface/glass_table report_as=text fields="_key,identifying_name,description,acl,definition" ```1```
| eval value=spath(value,"{}") 
| mvexpand value ```14 glass tables```
| eval glass_table_name = spath(value, "identifying_name") 
| search glass_table_name = star ```1 glass table```
| eval description = spath(value, "description") ```this is blank ???```
| eval key = spath(value, "_key") ```is this the glass table ky: 2913043e-76e1-11f0-abf2-0050568f9bbf```
| eval acl = spath(value, "acl") ```what is acl? an access control list listing what permissions are used on the glass table?```
| rex field=acl "\"owner\"\:\s\"(?<owner>\w+)\"" ```glass table owner/creator```
| eval definition = spath(value, "definition") ```what is definition?```
```| fields - value```
| eval ds_json = spath(definition, "dataSources") ```breakout data sources```

Thanks in advance for your assistance.

God bless,
Genesius

[victor_menezes]

@genesiusj 

Then you may have some opportunities to cleanup, but just combining my logic with yours current one, I have this:

| rest splunk_server=local /servicesNS/nobody/SA-ITOA/itoa_interface/glass_table report_as=text fields="_key,identifying_name,description,acl,definition" ```1```
| eval value=spath(value,"{}") 
| mvexpand value ```14 glass tables```
| eval glass_table_name = spath(value, "identifying_name") 
| search glass_table_name = "gex ecosystem - jp tes" ```1 glass table```
| eval description = spath(value, "description") ```this is blank ???```
| eval key = spath(value, "_key") ```is this the glass table ky: 2913043e-76e1-11f0-abf2-0050568f9bbf```
| eval acl = spath(value, "acl") ```what is acl? an access control list listing what permissions are used on the glass table?```
| rex field=acl "\"owner\"\:\s\"(?<owner>\w+)\"" ```glass table owner/creator```
| eval definition = spath(value, "definition") ```what is definition?```
| rex field="definition" max_match=0 "(?s)\"(?<ds_key>ds_[^\"]+)\"\\s*:\\s*\\{.*?\"name\"\\s*:\\s*\"(?<ds_name>[^\"]+)\""
| eval pair=mvzip(ds_key, ds_name, "||")
| mvexpand pair
| eval key=mvindex(split(pair,"||"),0),
name=mvindex(split(pair,"||"),1)
| table key name
| sort key

But you can append more values to it such as the GT name or even filtering it by name so your result will always be the one that you are filtering.

Also, to your questions there:

"Description" can be blank if malformed or not informed. If that is important to you, you can review your specific GTs and work on the extraction properly. I don't think it is as you never actually use it in your results in this example.

"acl" is an object containing access control list fields to map ownership, permissions and storage app of the object. It is a common object for Splunk knowledge objects. 

"definition" is the actual source code behind the GT, so there is where all the panels, queries, dropdowns, styles, etc are stored.