Splunk Search

Can I custom code a Splunk table to include JSON in a cell (without having the line feeds/endings removed)?

Path Finder

I have a Splunk application I am developing where I must put a pretty-print formatted JSON into the cell of a Splunk table for presenting to users. The table process flattens out the json removing line feeds / line endings. I would like to find a method likely involving HTML, CSS or javascript to keep the pretty-print type formatting within the cell and could use help and direction. I'm actually hoping someone has just done this. The dash board is put together with simple xml. Possible that simple xml extension will fix this. This is for Splunk 6.5 and higher.

0 Karma

SplunkTrust
SplunkTrust

@jimdiconectiv, as you would know that Splunk's EventViewerView is the only visualization element that can show pretty formatted jSON Data. You would have two options to pretty print format JSON. Both option require Simple XML JS Extention on top of Splunk JS Stack.

Option 1) Extend the CustomTableCellRenderer using JavaScript to handle pretty print formatting for column containing JSON Data. This would be time-consuming unless you have existing open source reusable JSON Data formatter using CSS and JS. There would still be time and effort required to plug in the CSS and JS files to Splunk.

Option 2) Create Table Row expansion to show EventViewer View with Raw JSON Data so that it is pretty print formatted.This is the option I will illustrate here as it can be coded in no time. It is based on Table Row Expansion example of Splunk Dashboard Examples App: https://splunkbase.splunk.com/app/1603/

alt text

Following are the steps used to create the dashboard:

Step 1) Add JSON Data (two sample events added below for reference). I have uploaded to index=main with sourcetype=_json in Splunk's default Search & Reporting App.

Step 2) Create a table showing jSON data. I have displayed only json nodes using | table header* requestCP*

Step 3) Give table an ID (i.e. id="expand_with_events" ) to be selected using SplunkJS mvc library function mvc.component.getInstance(expand_with_events)

Step 4) Create table_row_expansion_eventviewer_view.js file
a) Import tableview, eventviewer and searchmanager libraries (besides the other required libraries like mvc, underscore, simplexml/ready etc.).
b) Use table id expand_with_events to get the Table Element using SplunkJS mvc component function getInstance() based on id.
c) Call a Custom Table Row Expansion Render which creates a searchmanager and eventviewer in initialize section. PS: For setting up options for eventviewer refer to Splunk Web Framework documentation: http://docs.splunk.com/DocumentationStatic/WebFramework/1.1/compref_eventsviewer.html
d) Set the render() function to pick up a unique identifier from table row, in our case it is requestCP.ObjId. So that Event Viewer displays only one event on expanding the row.
e) Within render()function call the Search Manager created in initialize section to return raw event based on selected row ( requestCP.ObjId ).

Dashboard Code for reference:

<dashboard script="table_row_expansion_eventviewer_view.js">
  <label>jSON Data Formatting</label>
  <row>
    <panel>
      <table id="expand_with_events">
        <search>
          <query>index=main sourcetype="_json"
| table requestCP* header*</query>
          <earliest>0</earliest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">20</option>
      </table>
    </panel>
  </row>
</dashboard>

JavaScript File
table_row_expansion_eventviewer_view.js
code for reference:

require([
    'splunkjs/mvc/tableview',
    'splunkjs/mvc/eventsviewerview',
    'splunkjs/mvc/searchmanager',
    'splunkjs/mvc',
    'underscore',
    'splunkjs/mvc/simplexml/ready!'],function(
    TableView,
    EventView,
    SearchManager,
    mvc,
    _
    ){
    var EventSearchBasedRowExpansionRenderer = TableView.BaseRowExpansionRenderer.extend({
        initialize: function(args) {
            // initialize will run once, so we will set up a search and a chart to be reused.
            this._searchManager = new SearchManager({
                id: 'details-search-manager',
                preview: false
            });
            this._eventView = new EventView({
                managerid: 'details-search-manager',
                'list.drilldown': 'full'
            });
        },
        canRender: function(rowData) {
            // Since more than one row expansion renderer can be registered we let each decide if they can handle that
            // data
            // Here we will always handle it.
            return true;
        },
        render: function($container, rowData) {
            // rowData contains information about the row that is expanded.  We can see the cells, fields, and values
            // We will find the requestCP.ObjId cell to use its value
            var sourcetypeCell = _(rowData.cells).find(function (cell) {
               return cell.field === 'requestCP.ObjId';
            });
            //update the search with the requestCP.ObjId that we are interested in
            this._searchManager.set({ search: 'index=main sourcetype=_json requestCP.ObjId=' + sourcetypeCell.value});
            // $container is the jquery object where we can put out content.
            // In this case we will render our chart and add it to the $container
            $container.append(this._eventView.render().el);
        }
    });
    var tableElement = mvc.Components.getInstance("expand_with_events");
    tableElement.getVisualization(function(tableView) {
        // Add custom cell renderer, the table will re-render automatically.
        tableView.addRowExpansionRenderer(new EventSearchBasedRowExpansionRenderer());
    });
});

PS: I have used mocked data based on sample data provided in the question (following are the two JSON events data as displayed in the screenshot.)

Sample jSON Data for reference

Data 1

{ "header" : { "messageCategory" : "request", "messageType" : "MESSAGE_RESEND", "originationId" : "nightly-resend", "originationType" : "ADMIN", "receivedTs" : "2017-10-30T03:30:01.025Z" }, "requestCP" : { "ObjId" : 1770418, "targetList" : [ { "serviceId" : "cap5" } ] } }

Data 2

{ "header" : { "messageCategory" : "request", "messageType" : "MESSAGE_RESEND", "originationId" : "nightly-resend", "originationType" : "ADMIN", "receivedTs" : "2017-11-30T03:30:03.012Z" }, "requestCP" : { "ObjId" : 1770419, "targetList" : [ { "serviceId" : "cap5" } ] } }

PS: Since this example requires JavaScript. You may need to refresh/bump your Splunk instance for changes to reflect in your dashboard. May also required internet browser history cleanup.

This is not a run anywhere dashboard. So add the sample JSON Data (KV_MODE=json) and adjust your search accordingly.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Path Finder

Thanks, will be looking it over later today.

0 Karma

SplunkTrust
SplunkTrust

@jimdiconectiv , sure try out and confirm. Don't forget to download Splunk Dashboard Examples App from Splunkbase https://splunkbase.splunk.com/app/1603/ and try out the Table Row Expansion example as well. That will give you a feel of how this solution to show JSON Data on expanding a row will look like. Alternatively, when you search _raw events Event Viewer by default has this kind of inline drilldown where row expands to show related information for the row selected.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

SplunkTrust
SplunkTrust

If you just want Line Break for JSON Data in the table you can refer to @Ayn 's answer using CSS Style white-space:pre-line; option https://answers.splunk.com/answers/41691/add-newline-into-table-cell.html

Following is a run anywhere dashboard example for a table with id=table1, so that CSS Style override applies only to that table:

alt text

Simple XML Dashboard code (run anywhere query with mock data derived from the question):

<dashboard>
  <label>JSON Data format</label>
  <row>
    <panel>
      <html depends="$alwaysHideCSSStyle$">
        <style>
          #table1 td{
            white-space:pre-line;
          }
        </style>
      </html>
      <table id="table1">
        <search>
          <query>|  makeresults
|  eval _raw="{
\"header\" : {
\"messageCategory\" : \"request\",
\"messageType\" : \"MESSAGE_RESEND\",
\"originationId\" : \"nightly-resend\",
\"originationType\" : \"ADMIN\",
\"receivedTs\" : \"2017-10-30T03:30:01.025Z\"
},
\"requestCP\" : {
\"ObjId\" : 1770418,
\"targetList\" : [ {
\"serviceId\" : \"cap5\"
} ]
}
}"
| spath</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</dashboard>

PS: I have formatted your JSON in text-editor so that I can create a table with JSON data and run spath. However, if your objective is just to add new line characters the above CSS style override should still work for you. It is up to you whether you want to format JSON data or not.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Path Finder

This is helpful but we are still not quite in alignment on the basic problem. The table field I am concerned with already has newline characters similar to the _raw you created. The issue is that the table renderer in creating a web view of the table temporarily removes or ignores these new lines so the JSON looses it's pretty print formatting.

example as the cell might look in the table:
{ "header" : { "initiatorSuppression" : false, "Id" : 311048, "messageType" : "REPLY", "originationId" : "vxx", "originationTs" : "2017-10-30T03:30:06.828Z", "originationType" : "XML", "primaryCust" : "XHP3", "receivedTs" : "2017-10-30T03:30:06.886Z" }, "Reply" : { "Type" : "ObjectAttributeValueChange", "xmlBasicCode" : "success" } }
just wrapped, not pretty printed. The answer you referenced adds some space before the cell, cool, but not my main issue. I want this change is for readability. Too many different fields in the JSONS to code for how to specifically handle them. Short term, I do not need to go from JSON field to Splunk field by using spath.

Sorry I would have send screen shots, but some stuff is proprietary.

Basically, I just need the dispaly to honor formatting already there.

Thanks again.

0 Karma

SplunkTrust
SplunkTrust

@jimdiconectiv, by any chance can you add the jSON sample data. Also what is the content that you want to remove? Adding screenshot of what you have and what you want to remove would be quite helpful. Are you running a Splunk Search to get the data or REST API?

You can also consider creating your own custom visualization using any JavaScript/HTML/CSS based Table as per your need. However, for us to assist you better with what exactly you need you would need us to provide more details.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Path Finder

Niket, anyone,
Here is sample data with the json pretty print:
"{
""header"" : {
""messageCategory"" : ""request"",
""messageType"" : ""MESSAGE_RESEND"",
""originationId"" : ""nightly-resend"",
""originationType"" : ""ADMIN"",
""receivedTs"" : ""2017-10-30T03:30:01.025Z""
},
""requestCP"" : {
""ObjId"" : 1770418,
""targetList"" : [ {
""serviceId"" : ""cap5""
} ]
}
}"

There are some extra quotes there due to csv export. So in this case, I would want to see the JSON this way in a table cell.

The data export to csv files this way. So the formatting via crlf is still in the table data, just supressed.
The code for the table just flattens this into a long unpretty line that is folded and that's what I need to
fix.

I don't want to remove any content, just to have the cr/lf that give the JSON formatting its structure
maintained in table display. Probably with html/css / java script modification. The data is from a simple XML dashboard. Just looking for help, or possibly someone that has already done this.

Thanks!

Jim

0 Karma