topic Re: How to Create Multiple Tables in One Dashboard? in Splunk Search

How to Create Multiple Tables in One Dashboard?

gsmith93 — Wed, 20 Sep 2023 20:02:51 GMT

I am trying to create a Dashboard that hold multiple table of WebSphere App Server configuration data. The data I have looks like this:

{"ObjectType ":"AppServer","Object":"HJn6server1","Order":"147","Env":"UAT","SectionName":"Transport chain: WCInboundDefaultSecure:Channel HTTP", "Attributes":{"discriminationWeight": "10","enableLogging": "FALSE","keepAlive": "TRUE","maxFieldSize": "32768","maxHeaders": "500","maxRequestMessageBodySize": "-1","maximumPersistentRequests": "100","name": "HTTP_4","persistentTimeout": "30","readTimeout": "60","useChannelAccessLoggingSettings": "FALSE","useChannelErrorLoggingSettings": "FALSE","useChannelFRCALoggingSettings": "FALSE","writeTimeout": "60"}}

Where every event is a configuration section within an appserver where:

ObjectType - AppServer

Object - Name of Appserver (ex. "HJn6server1")

Env - Environment. (ex. Test, UAT, PROD)

SectionName - name within the appserver configuration that holds attributes.

Attributes - configuration attributes for a SectionName

I have been able to create one table per SectionName, but can't extend that to multiple sections.

I used the following code to make one table:

index = websphere_cct (Object= "HJn5server1" Env="Prod") OR (Object = "HJn7server3" Env="UAT") SectionName="Process Definition" Order
[ search index=websphere_cct SectionName
| dedup Order
| table Order ]
| fields - _*
| fields Object Attributes.* SectionName
| eval Object = ltrim(Object, " ")
| rename Attributes.* AS *
| table SectionName Object *
| fillnull value=""
| transpose column_name=Attribute header_field=Object
| eval match = if('HJn5server1' == 'HJn7server3', "y", "n")

Output:

Attribute HJn7server3 HJn5server1 match

SectionName	Process Definition	Process Definition	y
IBM_HEAPDUMP_OUTOFMEMORY			y
executableArguments	[]	[]	y
executableTarget	com.ibm.ws.runtime.WsServer	com.ibm.ws.runtime.WsServer	y
executableTargetKind	JAVA_CLASS	JAVA_CLASS	y
startCommandArgs	[]	[]	y
stopCommandArgs	[]	[]	y
terminateCommandArgs	[]	[]	y
workingDirectory	${USER_INSTALL_ROOT}	${USER_INSTALL_ROOT}	y

What I would like to do is to create as many tables as there are SectionNames for a given comparison between two Objects.

But I cannot figure out how to modify the code for allowing to have several tables in one dashboard for multiple SectionNames with their associated Attributes for two appservers in comparison.

Please help.

Re: How to Create Multiple Tables in One Dashboard?

ITWhisperer — Thu, 21 Sep 2023 09:05:46 GMT

It seems to me that you need the section name to be part of the attribute name. Try something like this

| foreach Attributes.* [| eval name=SectionName.".<<MATCHSEG1>>" | eval {name}='<<FIELD>>'] | fields - Attributes.* name SectionName | fillnull value="" | transpose column_name=Attribute header_field=Object | eval match = if('HJn5server1' == 'HJn7server3', "y", "n")

Re: How to Create Multiple Tables in One Dashboard?

yuanliu — Thu, 21 Sep 2023 12:08:53 GMT

The feature you are looking for is trellis. But Splunk doesn't currently do trellis for table visualization. (I'm almost sure that Grafana does.) You can sort of hack something yourself if you are willing to get into the nitty-gritty Simple XML programming. (Or in Dashboard Studio source.) Oh, you also need to know all possible values of SectionName in advance. The basic idea is

Run a <query /> to populate an aggregate token with values of Attribute in the same search window, e.g.,

index = websphere_cct (Object= "HJn5server1" Env="Prod") OR (Object = "HJn7server3" Env="UAT") SectionName="Process Definition" | spath path=Attributes | eval Attributes = mvappend("SectionName", json_array_to_mv(json_keys(Attributes)))

(Note this only runs in Splunk 8.0 and later.)

Use <condition><progress /></condition> to set or unset a dedicated token for every possible SectionName value. If the value exist in that aggregate token, set the token, otherwise unset it.
Use these dedicated tokens to hide or show tables, one for each possible SectionName value.

You can read about hide-and-show in Access tokens to show or hide user interface components, about set dynamic tokens in Search tokens for dynamic display example.

Here is a mock dashboard you can play with. (I included comments about the actual search that you can substitute.) Alas! The code is too long. You can download/copy from here: Mock table trellis in Splunk Simple XML.

Here is a screenshot:

As you can see, from your illustrated attribute list of 9, my mock search pretends to have found 4. So, only those 4 corresponding trellis show on the left-hand side. If you edit the attribute selection (in source), different tables will show. In edit mode, all 9 tables are visible, with hidden ones in grey. (Right-hand side is the big table you illustrated, with all 9 attributes.)

Several notes:

You could have saved volunteers lots of time (and done yourself a favor) by illustrating sample data that matches your desired output. The JSON in the description has too little in common with the table you show.
As your search restricts SectionName to "Process Definition", it doesn't seem to make sense to list SectionName in the table. (SectionName is not an Attribute, any way.) But I still included it in my emulation.
Maintenance is painful and not very scalable like a true trellis feature.

Hope this helps.

Re: How to Create Multiple Tables in One Dashboard?

gsmith93 — Thu, 21 Sep 2023 18:35:00 GMT

yuanliu:

Thank you for the reply. As per having a illustrating sample data, I went over the character limit; however, I could have just made another post. Anyway, here is what I am looking for. It is muptiple tables in one dashboard where the tables are ordered by SectionName followed by the Attributes for that section and a column stating the attributes match or do not match.

I will look at the rest of your post now.

Re: How to Create Multiple Tables in One Dashboard?

gsmith93 — Thu, 21 Sep 2023 20:12:20 GMT

ITWhisperer:

Thank you for the reply. Combining the 'SectionName' with each attribute makes sense; however, I can't get your code to resolve correctly. Here are my results with the following code:

index = websphere_cct (Object= "HJn5server1" Env="UAT") OR (Object = "HJn2server1" Env="Prod")
| foreach Attributes.*
[| eval name=SectionName.".<<MATCHSEG1>>"
| eval {name}='<<FIELD>>']
| fields - Attributes.* name SectionName
| fillnull value=""
| transpose column_name=Attribute header_field=Object
| eval match = if('HJn5server1' == 'HJn7server3', "y", "n")

I can't seem to get the value of the "Attributes" field to printout.

Re: How to Create Multiple Tables in One Dashboard?

ITWhisperer — Thu, 21 Sep 2023 22:50:43 GMT

Try gathering the different fields by Object

| foreach Attributes.* [| eval name=SectionName.".<<MATCHSEG1>>" | eval {name}='<<FIELD>>'] | fields - Attributes.* name SectionName | stats values(*) as * by Object | transpose column_name=Attribute header_field=Object | eval match = if('HJn5server1' == 'HJn7server3', "y", "n")

Re: How to Create Multiple Tables in One Dashboard?

yuanliu — Sat, 23 Sep 2023 07:56:28 GMT

So, you want trellis breakdown on SectionName, not on Attribute. This says even more of the importance of illustrating data input and desired output. Really all of this should be in text. Anyway, I mocked up three SectionName values based off what you showed in both text and screenshots, and revised https://gist.github.com/whackyhack/af1d0b20e297a5237594cbb6aaacc0f6 to mock trellis by SectionName. Needless to say: You need to know all possible values of SectionName in advance. (And if you want the tables to be in alphabetic order, you need to arrange all of them in that order manually.) The token setting is based on a search where only two SectionName values are returned, namely HTTP Plugin settings and Transport chain: WCInboundDefaultSecure:Channel HTTP. Therefore table for Process Definition is hidden.

Core search to set token by SessionName should be simply

index = websphere_cct (Object= "HJn5server1" Env="Prod") OR (Object = "HJn7server3" Env="UAT") [ search index=websphere_cct SectionName=* | dedup Order | table Order ] | stats values(SectionName) as SectionName

Also note that token name cannot contain colon (:); for this reason, I used $Transport chain- WCInboundDefaultSecure-Channel HTTP$ as token name. Again this mock trellis requires you to know all possible values of SectionName in advance, and manually program all <table/>

For you to more easily compare to real data, here is the data emulation code

| makeresults | eval data = mvappend("{\"ObjectType \":\"AppServer\",\"Object\":\"HJn7server3\",\"Order\":\"147\",\"Env\":\"UAT\",\"SectionName\":\"Transport chain: WCInboundDefaultSecure:Channel HTTP\", \"Attributes\":{\"discriminationWeight\": \"10\",\"enableLogging\": \"FALSE\",\"keepAlive\": \"TRUE\",\"maxFieldSize\": \"32768\",\"maxHeaders\": \"500\",\"maxRequestMessageBodySize\": \"-1\",\"maximumPersistentRequests\": \"100\",\"name\": \"HTTP_4\",\"persistentTimeout\": \"30\",\"readTimeout\": \"60\",\"useChannelAccessLoggingSettings\": \"FALSE\",\"useChannelErrorLoggingSettings\": \"FALSE\",\"useChannelFRCALoggingSettings\": \"FALSE\",\"writeTimeout\": \"60\"}}", "{\"ObjectType \":\"AppServer\",\"Object\":\"HJn5server1\",\"Order\":\"147\",\"Env\":\"UAT\",\"SectionName\":\"Transport chain: WCInboundDefaultSecure:Channel HTTP\", \"Attributes\":{\"discriminationWeight\": \"10\",\"enableLogging\": \"FALSE\",\"keepAlive\": \"TRUE\",\"maxFieldSize\": \"32768\",\"maxHeaders\": \"500\",\"maxRequestMessageBodySize\": \"-1\",\"maximumPersistentRequests\": \"100\",\"name\": \"HTTP_4\",\"persistentTimeout\": \"30\",\"readTimeout\": \"60\",\"useChannelAccessLoggingSettings\": \"FALSE\",\"useChannelErrorLoggingSettings\": \"FALSE\",\"useChannelFRCALoggingSettings\": \"FALSE\",\"writeTimeout\": \"60\"}}", "{\"ObjectType \":\"AppServer\",\"Object\":\"HJn7server3\",\"Order\":\"147\",\"Env\":\"PROD\",\"SectionName\":\"Process Definition\", \"Attributes\":{\"IBM_HEAPDUMP_OUTOFMEMORY\": \"\",\"executableArguments\": \"[]\",\"executableTarget\": \"com.ibm.ws.runtime.WsServer\",\"executableTargetKind\": \"JAVA_CLASS\",\"startCommandArgs\": \"[]\",\"stopCommandArgs\": \"[]\",\"terminateCommandArgs\": \"[]\",\"workingDirectory\": \"${USER_INSTALL_ROOT}\"}}", "{\"ObjectType \":\"AppServer\",\"Object\":\"HJn5server1\",\"Order\":\"147\",\"Env\":\"UAT\",\"SectionName\":\"Process Definition\", \"Attributes\":{\"IBM_HEAPDUMP_OUTOFMEMORY\": \"\",\"executableArguments\": \"[]\",\"executableTarget\": \"com.ibm.ws.runtime.WsServer\",\"executableTargetKind\": \"JAVA_CLASS\",\"startCommandArgs\": \"[]\",\"stopCommandArgs\": \"[]\",\"terminateCommandArgs\": \"[]\",\"workingDirectory\": \"${USER_INSTALL_ROOT}\"}}", "{\"ObjectType \":\"AppServer\",\"Object\":\"HJn7server3\",\"Order\":\"147\",\"Env\":\"PROD\",\"SectionName\":\"HTTP Plugin settings\", \"Attributes\":{\"ConnectTimeout\": 5,\"MaxConnections\": -1,\"Role\": \"PRIMARY\",\"ExtendedHandshake\": false,\"ServerIOTimeout\": 900,\"waitForContinue\": false}}", "{\"ObjectType \":\"AppServer\",\"Object\":\"HJn5server1\",\"Order\":\"147\",\"Env\":\"PROD\",\"SectionName\":\"HTTP Plugin settings\", \"Attributes\":{\"ConnectTimeout\": 5,\"MaxConnections\": -1,\"Role\": \"PRIMARY\",\"ExtendedHandshake\": false,\"ServerIOTimeout\": 900,\"waitForContinue\": false}}") | mvexpand data | rename data as _raw ``` above emulates index = websphere_cct (Object= "HJn5server1" Env="Prod") OR (Object = "HJn7server3" Env="UAT") [ search index=websphere_cct SectionName=* | dedup Order | table Order ] ```

_raw

{"ObjectType ":"AppServer","Object":"HJn7server3","Order":"147","Env":"UAT","SectionName":"Transport chain: WCInboundDefaultSecure:Channel HTTP", "Attributes":{"discriminationWeight": "10","enableLogging": "FALSE","keepAlive": "TRUE","maxFieldSize": "32768","maxHeaders": "500","maxRequestMessageBodySize": "-1","maximumPersistentRequests": "100","name": "HTTP_4","persistentTimeout": "30","readTimeout": "60","useChannelAccessLoggingSettings": "FALSE","useChannelErrorLoggingSettings": "FALSE","useChannelFRCALoggingSettings": "FALSE","writeTimeout": "60"}}

{"ObjectType ":"AppServer","Object":"HJn5server1","Order":"147","Env":"UAT","SectionName":"Transport chain: WCInboundDefaultSecure:Channel HTTP", "Attributes":{"discriminationWeight": "10","enableLogging": "FALSE","keepAlive": "TRUE","maxFieldSize": "32768","maxHeaders": "500","maxRequestMessageBodySize": "-1","maximumPersistentRequests": "100","name": "HTTP_4","persistentTimeout": "30","readTimeout": "60","useChannelAccessLoggingSettings": "FALSE","useChannelErrorLoggingSettings": "FALSE","useChannelFRCALoggingSettings": "FALSE","writeTimeout": "60"}}

{"ObjectType ":"AppServer","Object":"HJn7server3","Order":"147","Env":"PROD","SectionName":"Process Definition", "Attributes":{"IBM_HEAPDUMP_OUTOFMEMORY": "","executableArguments": "[]","executableTarget": "com.ibm.ws.runtime.WsServer","executableTargetKind": "JAVA_CLASS","startCommandArgs": "[]","stopCommandArgs": "[]","terminateCommandArgs": "[]","workingDirectory": "${USER_INSTALL_ROOT}"}}

{"ObjectType ":"AppServer","Object":"HJn5server1","Order":"147","Env":"UAT","SectionName":"Process Definition", "Attributes":{"IBM_HEAPDUMP_OUTOFMEMORY": "","executableArguments": "[]","executableTarget": "com.ibm.ws.runtime.WsServer","executableTargetKind": "JAVA_CLASS","startCommandArgs": "[]","stopCommandArgs": "[]","terminateCommandArgs": "[]","workingDirectory": "${USER_INSTALL_ROOT}"}}

{"ObjectType ":"AppServer","Object":"HJn7server3","Order":"147","Env":"PROD","SectionName":"HTTP Plugin settings", "Attributes":{"ConnectTimeout": 5,"MaxConnections": -1,"Role": "PRIMARY","ExtendedHandshake": false,"ServerIOTimeout": 900,"waitForContinue": false}}

{"ObjectType ":"AppServer","Object":"HJn5server1","Order":"147","Env":"PROD","SectionName":"HTTP Plugin settings", "Attributes":{"ConnectTimeout": 5,"MaxConnections": -1,"Role": "PRIMARY","ExtendedHandshake": false,"ServerIOTimeout": 900,"waitForContinue": false}}

Re: How to Create Multiple Tables in One Dashboard?

gsmith93 — Mon, 25 Sep 2023 20:42:33 GMT

ITWisperer:

Thank you so much for the response.

I believe what you gave me will do the job; however, I have some questions. Here is the code I used and how it turned out. For expediency, I just used one 'SectionName' ... this field holds the name of the section in WebSphere that holds the attributes of a section of the Application Server.

index = websphere_cct (Object= "HJn3server2" Env="Prod") OR (Object = "HJn8server3" Env="UAT") SectionName="JVM Configuration"
| foreach Attributes.*
[| eval name=SectionName.".<<MATCHSEG1>>"
| eval {name}='<<FIELD>>']
| fields - Attributes.* name SectionName
| stats values(*) as * by Object
| transpose column_name='SectionName.Attribute' header_field=Object
| eval match = if('HJn3server2' == 'HJn8server3', "y", "n")

And here are the results:

My questions are:

1) How does the command 'stats values(*) as * by Object' work? The 'Object' field is the name of the application server is this case. How does that command group the values of the Attributes by Object?

2) Why are there so many extra fields in the table? Such as: Order, OrderType, Index, etc (per below)

Why do these get included and is the only way to get rid of these fields is thru the 'fields' command? (ie fields - Order - Ordertype - Index ..)

Thank you very much for the help !!!

Re: How to Create Multiple Tables in One Dashboard?

ITWhisperer — Mon, 25 Sep 2023 22:06:02 GMT

You might want to consider just keeping a couple of the fields along with the Atrributes.* fields

index = websphere_cct (Object= "HJn3server2" Env="Prod") OR (Object = "HJn8server3" Env="UAT") SectionName="JVM Configuration" | table Attributes.* SectionName Object Env | foreach Attributes.* [| eval name=SectionName.".<<MATCHSEG1>>" | eval {name}='<<FIELD>>'] | fields - Attributes.* name SectionName | stats values(*) as * by Object | transpose column_name='SectionName.Attribute' header_field=Object | eval match = if('HJn3server2' == 'HJn8server3', "y", "n")

The stats values(*) as * by Object will put all the values of all the fields (which don't start with _) in the same row for the same Object field value.