Getting Data In

How to add free text to table?

oh_my_lawdy
Explorer

Hello!

I have a dataset that I'd like to add a new field to where I can arbitrarily define the values with manual input without downloading and reuploading the data. I've tried editing the table but it seems as though I can only enter a calculated value, some cacatenation of fields and values, or input the same value for every record. Any help is appreciated, thanks!

example:

original dataset

OG Field 1OG Field 2OG Field 3
UUIDtimestampvalue
UUIDtimestampvalue

 

new dataset

OG Field 1OG Field 2OG Field 3New Field
UUIDtimestampvalueI can input anything I want here like a comment on the record
UUIDtimestampvalueI can input something different here

 

I don't necessarily need to use tables so if there's another method of adding new fields to datasets from within Splunk I'm open to that as well.

Labels (1)
0 Karma
1 Solution

kamlesh_vaghela
SplunkTrust
SplunkTrust

@oh_my_lawdy 

I also suggest the same way as @richgalloway .

Please check below sample code for your requirement. 

For KV Store Lookup. Here, you can change the configuration per your requirement.

collections.conf

[kvstoreog]
enforceTypes = true
field.new_field = string

transforms.conf

[og_info]
external_type = kvstore
case_sensitive_match = true
collection = kvstoreog
fields_list = _key, new_field

 

Dashboard XML. Kindly change search and the key value as per your requirement.

 

<dashboard script="a.js">
  <label>add free text to table</label>
  <row>
    <panel>
      <table id="tbl1">
        <search>
          <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3
0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1
68a521188b984d2b84e41d35e4ceffa9,timestamp,value2
e646271ee11c4264ba4747519f1ba0e5,timestamp,value3
925fd4c2dc27413384f1a51c11721864,timestamp,value4
b8a9848134a243b684a44e4895da6c8f,timestamp,value5
jjj9848134a243b684a44e4895da6c8f,timestamp,value6
" | multikv forceheader=1
| eval a=10 | accum a
| eval n=1648188616
| eval OG_Field_2 = n - a
| table OG_Field_1 OG_Field_2 OG_Field_3
| rename comment as "Upto now is for sample data only"
| eval New_Field = OG_Field_1."_".OG_Field_2</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <search id="search1">
          <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3
0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1
68a521188b984d2b84e41d35e4ceffa9,timestamp,value2
e646271ee11c4264ba4747519f1ba0e5,timestamp,value3
925fd4c2dc27413384f1a51c11721864,timestamp,value4
b8a9848134a243b684a44e4895da6c8f,timestamp,value5
jjj9848134a243b684a44e4895da6c8f,timestamp,value6
" | multikv forceheader=1
| eval a=10 | accum a
| eval n=1648188616
| eval OG_Field_2 = n - a
| table OG_Field_1 OG_Field_2 OG_Field_3
| rename comment as "Upto now is for sample data only"
| eval data_key = OG_Field_1."_".OG_Field_2
| lookup og_info _key as data_key</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</dashboard>

 

a.js

require([
    'underscore',
    'jquery',
    'splunkjs/mvc',
    'splunkjs/mvc/tableview',
    'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
    console.log("LAZY!!!");
    var service = mvc.createService({ owner: "nobody" });
    var mySearch = mvc.Components.get("search1");
    // Add overlay buttons to table
    var CustomRangeRenderer = TableView.BaseCellRenderer.extend({
        canRender: function(cell) {
            return _(["New_Field"]).contains(cell.field);
        },
        render: function($td, cell) {
            if (cell.field === "New_Field") {
                var strHtmlInput = '<input type="text">';
                $td.append(strHtmlInput).on("change", function(e) {
                    console.log(e.target.value)
                    console.log("Saved", (new Date).getTime())
                        // Provide Record to store 
                    var record = {
                        "_key": cell.value,
                        "new_field": e.target.value
                    };
                    service.request("storage/collections/data/kvstoreog/" + cell.value, "GET", null, null, null, { "Content-Type": "application/json" }, null).done(function(response) {
                        console.log(response);
                        service.request("storage/collections/data/kvstoreog/" + cell.value, "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() {
                            console.log("Update Done");
                            mySearch.startSearch();
                        });
                    }).error(function(err) {
                        console.log(err.responseText);
                        service.request("storage/collections/data/kvstoreog/", "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() {
                            console.log("Insert Done");
                            mySearch.startSearch();
                        });
                    });
                });
            }
        }
    });

    var sh = mvc.Components.get("tbl1");
    if (typeof(sh) != "undefined") {
        sh.getVisualization(function(tableView) {
            // Add custom cell renderer and force re-render
            tableView.table.addCellRenderer(new CustomRangeRenderer());
            tableView.table.render();
        });
    }
});

 

Screen

Screenshot 2022-03-25 at 9.09.03 PM.png

 

To get batter understanding, please use same configuration and code for POC and do change to play with it.

 

Thanks
KV


If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated. 

 

 

 

View solution in original post

kamlesh_vaghela
SplunkTrust
SplunkTrust

@oh_my_lawdy 

I also suggest the same way as @richgalloway .

Please check below sample code for your requirement. 

For KV Store Lookup. Here, you can change the configuration per your requirement.

collections.conf

[kvstoreog]
enforceTypes = true
field.new_field = string

transforms.conf

[og_info]
external_type = kvstore
case_sensitive_match = true
collection = kvstoreog
fields_list = _key, new_field

 

Dashboard XML. Kindly change search and the key value as per your requirement.

 

<dashboard script="a.js">
  <label>add free text to table</label>
  <row>
    <panel>
      <table id="tbl1">
        <search>
          <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3
0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1
68a521188b984d2b84e41d35e4ceffa9,timestamp,value2
e646271ee11c4264ba4747519f1ba0e5,timestamp,value3
925fd4c2dc27413384f1a51c11721864,timestamp,value4
b8a9848134a243b684a44e4895da6c8f,timestamp,value5
jjj9848134a243b684a44e4895da6c8f,timestamp,value6
" | multikv forceheader=1
| eval a=10 | accum a
| eval n=1648188616
| eval OG_Field_2 = n - a
| table OG_Field_1 OG_Field_2 OG_Field_3
| rename comment as "Upto now is for sample data only"
| eval New_Field = OG_Field_1."_".OG_Field_2</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <search id="search1">
          <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3
0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1
68a521188b984d2b84e41d35e4ceffa9,timestamp,value2
e646271ee11c4264ba4747519f1ba0e5,timestamp,value3
925fd4c2dc27413384f1a51c11721864,timestamp,value4
b8a9848134a243b684a44e4895da6c8f,timestamp,value5
jjj9848134a243b684a44e4895da6c8f,timestamp,value6
" | multikv forceheader=1
| eval a=10 | accum a
| eval n=1648188616
| eval OG_Field_2 = n - a
| table OG_Field_1 OG_Field_2 OG_Field_3
| rename comment as "Upto now is for sample data only"
| eval data_key = OG_Field_1."_".OG_Field_2
| lookup og_info _key as data_key</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</dashboard>

 

a.js

require([
    'underscore',
    'jquery',
    'splunkjs/mvc',
    'splunkjs/mvc/tableview',
    'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
    console.log("LAZY!!!");
    var service = mvc.createService({ owner: "nobody" });
    var mySearch = mvc.Components.get("search1");
    // Add overlay buttons to table
    var CustomRangeRenderer = TableView.BaseCellRenderer.extend({
        canRender: function(cell) {
            return _(["New_Field"]).contains(cell.field);
        },
        render: function($td, cell) {
            if (cell.field === "New_Field") {
                var strHtmlInput = '<input type="text">';
                $td.append(strHtmlInput).on("change", function(e) {
                    console.log(e.target.value)
                    console.log("Saved", (new Date).getTime())
                        // Provide Record to store 
                    var record = {
                        "_key": cell.value,
                        "new_field": e.target.value
                    };
                    service.request("storage/collections/data/kvstoreog/" + cell.value, "GET", null, null, null, { "Content-Type": "application/json" }, null).done(function(response) {
                        console.log(response);
                        service.request("storage/collections/data/kvstoreog/" + cell.value, "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() {
                            console.log("Update Done");
                            mySearch.startSearch();
                        });
                    }).error(function(err) {
                        console.log(err.responseText);
                        service.request("storage/collections/data/kvstoreog/", "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() {
                            console.log("Insert Done");
                            mySearch.startSearch();
                        });
                    });
                });
            }
        }
    });

    var sh = mvc.Components.get("tbl1");
    if (typeof(sh) != "undefined") {
        sh.getVisualization(function(tableView) {
            // Add custom cell renderer and force re-render
            tableView.table.addCellRenderer(new CustomRangeRenderer());
            tableView.table.render();
        });
    }
});

 

Screen

Screenshot 2022-03-25 at 9.09.03 PM.png

 

To get batter understanding, please use same configuration and code for POC and do change to play with it.

 

Thanks
KV


If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated. 

 

 

 

oh_my_lawdy
Explorer

Hi @kamlesh_vaghela , I've been experimenting with the dashboard and script you provided, but my dashboard isn't allow for free text input and instead calculates the `New_Field` value as 

OG_Field_1 + '_' + OG_Field_2

See screenshot

oh_my_lawdy_0-1648238456251.png

I've added the .conf files to $SPLUNK_HOME/etc/apps/search/local and a.js to $SPLUNK_HOME/etc/apps/search/appserver/static/ and then I created a new dashboard with your XML source code. I then restarted the server.

Is this the correct setup?

 

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@oh_my_lawdy 

May be we need to bump the javascript.. Just open this link and click on bump button.

https://localhost:8000/en-US/_bump

Open dashboard and right click on dashboard- open Inspect Element and check any error in console?

Screenshot 2022-03-28 at 10.50.59 AM.png

 If you found any error please share to us.

 

KV

 

oh_my_lawdy
Explorer

That fixed it, thanks!

0 Karma

oh_my_lawdy
Explorer

@kamlesh_vaghela , thank you so much! This example is exactly what I needed

0 Karma

PickleRick
SplunkTrust
SplunkTrust

I'm not sure what you mean by "dataset" here. Within Splunk it has a very specific meaning pertaining to data models functionality and I'm not entirely sure that's what you mean.

Can you elaborate more precisely what you want to achieve?

As a general rule you can't edit (modify) already indexed data. You can use various search-time methods to calculate dynamicaly created values but they generally, as you noticed, rely on the data you already have. And working in search-time they don't modify the data present i  indexes, they just return to you the data in "recalculated" form.

oh_my_lawdy
Explorer

ahh okay, thanks for the good info. I'm pretty new to using Splunk and learning as I go. 

I'm wanting to label data for training an MLTK algorithm from within Splunk. The data is a timeseries so reach goal is to chart the data and visually select points where a label should be. Alternatively, I'd like to just be able to label the individual records somehow, maybe with a lookup table?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Everyone was new once 🙂

And yes, you can create a lookup from which you'd retrieve labels for your data points. But you need some field or combination of fields to look up by.

Be warned thought that if you have too much data labeled you'll efectively get into territory of effectively duplicating quite a lot of your events into said lookup. So it might be worth considering labeling the data before ingesting it into splunk.

richgalloway
SplunkTrust
SplunkTrust

Datasets cannot be modified.  You'll have to store the comments in a lookup table (consider the KVstore) and somehow link it to the dataset.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...