How to add free text to table?

[oh_my_lawdy]

Hello!

I have a dataset that I'd like to add a new field to where I can arbitrarily define the values with manual input without downloading and reuploading the data. I've tried editing the table but it seems as though I can only enter a calculated value, some cacatenation of fields and values, or input the same value for every record. Any help is appreciated, thanks!

example:

original dataset

OG Field 1	OG Field 2	OG Field 3
UUID	timestamp	value
UUID	timestamp	value

new dataset

OG Field 1	OG Field 2	OG Field 3	New Field
UUID	timestamp	value	I can input anything I want here like a comment on the record
UUID	timestamp	value	I can input something different here

I don't necessarily need to use tables so if there's another method of adding new fields to datasets from within Splunk I'm open to that as well.

[kamlesh_vaghela · Accepted solution]

@oh_my_lawdy 

I also suggest the same way as @richgalloway .

Please check below sample code for your requirement. 

For KV Store Lookup. Here, you can change the configuration per your requirement.

collections.conf

[kvstoreog]
enforceTypes = true
field.new_field = string

transforms.conf

[og_info]
external_type = kvstore
case_sensitive_match = true
collection = kvstoreog
fields_list = _key, new_field

Dashboard XML. Kindly change search and the key value as per your requirement.

<dashboard script="a.js">
  <label>add free text to table</label>
  <row>
    <panel>
      <table id="tbl1">
        <search>
          <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3
0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1
68a521188b984d2b84e41d35e4ceffa9,timestamp,value2
e646271ee11c4264ba4747519f1ba0e5,timestamp,value3
925fd4c2dc27413384f1a51c11721864,timestamp,value4
b8a9848134a243b684a44e4895da6c8f,timestamp,value5
jjj9848134a243b684a44e4895da6c8f,timestamp,value6
" | multikv forceheader=1
| eval a=10 | accum a
| eval n=1648188616
| eval OG_Field_2 = n - a
| table OG_Field_1 OG_Field_2 OG_Field_3
| rename comment as "Upto now is for sample data only"
| eval New_Field = OG_Field_1."_".OG_Field_2</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <search id="search1">
          <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3
0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1
68a521188b984d2b84e41d35e4ceffa9,timestamp,value2
e646271ee11c4264ba4747519f1ba0e5,timestamp,value3
925fd4c2dc27413384f1a51c11721864,timestamp,value4
b8a9848134a243b684a44e4895da6c8f,timestamp,value5
jjj9848134a243b684a44e4895da6c8f,timestamp,value6
" | multikv forceheader=1
| eval a=10 | accum a
| eval n=1648188616
| eval OG_Field_2 = n - a
| table OG_Field_1 OG_Field_2 OG_Field_3
| rename comment as "Upto now is for sample data only"
| eval data_key = OG_Field_1."_".OG_Field_2
| lookup og_info _key as data_key</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</dashboard>

a.js

require([
    'underscore',
    'jquery',
    'splunkjs/mvc',
    'splunkjs/mvc/tableview',
    'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
    console.log("LAZY!!!");
    var service = mvc.createService({ owner: "nobody" });
    var mySearch = mvc.Components.get("search1");
    // Add overlay buttons to table
    var CustomRangeRenderer = TableView.BaseCellRenderer.extend({
        canRender: function(cell) {
            return _(["New_Field"]).contains(cell.field);
        },
        render: function($td, cell) {
            if (cell.field === "New_Field") {
                var strHtmlInput = '<input type="text">';
                $td.append(strHtmlInput).on("change", function(e) {
                    console.log(e.target.value)
                    console.log("Saved", (new Date).getTime())
                        // Provide Record to store 
                    var record = {
                        "_key": cell.value,
                        "new_field": e.target.value
                    };
                    service.request("storage/collections/data/kvstoreog/" + cell.value, "GET", null, null, null, { "Content-Type": "application/json" }, null).done(function(response) {
                        console.log(response);
                        service.request("storage/collections/data/kvstoreog/" + cell.value, "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() {
                            console.log("Update Done");
                            mySearch.startSearch();
                        });
                    }).error(function(err) {
                        console.log(err.responseText);
                        service.request("storage/collections/data/kvstoreog/", "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() {
                            console.log("Insert Done");
                            mySearch.startSearch();
                        });
                    });
                });
            }
        }
    });

    var sh = mvc.Components.get("tbl1");
    if (typeof(sh) != "undefined") {
        sh.getVisualization(function(tableView) {
            // Add custom cell renderer and force re-render
            tableView.table.addCellRenderer(new CustomRangeRenderer());
            tableView.table.render();
        });
    }
});

Screen

To get batter understanding, please use same configuration and code for POC and do change to play with it.

Thanks
KV


If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated. 

[oh_my_lawdy]

Hi @kamlesh_vaghela , I've been experimenting with the dashboard and script you provided, but my dashboard isn't allow for free text input and instead calculates the `New_Field` value as 

OG_Field_1 + '_' + OG_Field_2

See screenshot

I've added the .conf files to $SPLUNK_HOME/etc/apps/search/local and a.js to $SPLUNK_HOME/etc/apps/search/appserver/static/ and then I created a new dashboard with your XML source code. I then restarted the server.

Is this the correct setup?

[kamlesh_vaghela]

@oh_my_lawdy 

May be we need to bump the javascript.. Just open this link and click on bump button.

https://localhost:8000/en-US/_bump

Open dashboard and right click on dashboard- open Inspect Element and check any error in console?

 If you found any error please share to us.

KV

[oh_my_lawdy]

That fixed it, thanks!

[oh_my_lawdy]

@kamlesh_vaghela , thank you so much! This example is exactly what I needed

[PickleRick]

I'm not sure what you mean by "dataset" here. Within Splunk it has a very specific meaning pertaining to data models functionality and I'm not entirely sure that's what you mean.

Can you elaborate more precisely what you want to achieve?

As a general rule you can't edit (modify) already indexed data. You can use various search-time methods to calculate dynamicaly created values but they generally, as you noticed, rely on the data you already have. And working in search-time they don't modify the data present i  indexes, they just return to you the data in "recalculated" form.

[oh_my_lawdy]

ahh okay, thanks for the good info. I'm pretty new to using Splunk and learning as I go. 

I'm wanting to label data for training an MLTK algorithm from within Splunk. The data is a timeseries so reach goal is to chart the data and visually select points where a label should be. Alternatively, I'd like to just be able to label the individual records somehow, maybe with a lookup table?

[PickleRick]

Everyone was new once 🙂

And yes, you can create a lookup from which you'd retrieve labels for your data points. But you need some field or combination of fields to look up by.

Be warned thought that if you have too much data labeled you'll efectively get into territory of effectively duplicating quite a lot of your events into said lookup. So it might be worth considering labeling the data before ingesting it into splunk.

[richgalloway]

Datasets cannot be modified.  You'll have to store the comments in a lookup table (consider the KVstore) and somehow link it to the dataset.