topic Re: How to add free text to table? in Getting Data In

How to add free text to table?

oh_my_lawdy — Thu, 24 Mar 2022 15:44:17 GMT

Hello!

I have a dataset that I'd like to add a new field to where I can arbitrarily define the values with manual input without downloading and reuploading the data. I've tried editing the table but it seems as though I can only enter a calculated value, some cacatenation of fields and values, or input the same value for every record. Any help is appreciated, thanks!

example:

original dataset

OG Field 1	OG Field 2	OG Field 3
UUID	timestamp	value
UUID	timestamp	value

new dataset

OG Field 1	OG Field 2	OG Field 3	New Field
UUID	timestamp	value	I can input anything I want here like a comment on the record
UUID	timestamp	value	I can input something different here

I don't necessarily need to use tables so if there's another method of adding new fields to datasets from within Splunk I'm open to that as well.

Re: How to add free text to table?

richgalloway — Thu, 24 Mar 2022 16:53:40 GMT

Datasets cannot be modified. You'll have to store the comments in a lookup table (consider the KVstore) and somehow link it to the dataset.

Re: How to add free text to table?

PickleRick — Thu, 24 Mar 2022 20:46:40 GMT

I'm not sure what you mean by "dataset" here. Within Splunk it has a very specific meaning pertaining to data models functionality and I'm not entirely sure that's what you mean.

Can you elaborate more precisely what you want to achieve?

As a general rule you can't edit (modify) already indexed data. You can use various search-time methods to calculate dynamicaly created values but they generally, as you noticed, rely on the data you already have. And working in search-time they don't modify the data present i indexes, they just return to you the data in "recalculated" form.

Re: How to add free text to table?

oh_my_lawdy — Thu, 24 Mar 2022 21:37:33 GMT

ahh okay, thanks for the good info. I'm pretty new to using Splunk and learning as I go.

I'm wanting to label data for training an MLTK algorithm from within Splunk. The data is a timeseries so reach goal is to chart the data and visually select points where a label should be. Alternatively, I'd like to just be able to label the individual records somehow, maybe with a lookup table?

Re: How to add free text to table?

PickleRick — Thu, 24 Mar 2022 21:54:35 GMT

Everyone was new once 🙂

And yes, you can create a lookup from which you'd retrieve labels for your data points. But you need some field or combination of fields to look up by.

Be warned thought that if you have too much data labeled you'll efectively get into territory of effectively duplicating quite a lot of your events into said lookup. So it might be worth considering labeling the data before ingesting it into splunk.

Re: How to add free text to table?

kamlesh_vaghela — Fri, 25 Mar 2022 15:40:27 GMT

@oh_my_lawdy

I also suggest the same way as @richgalloway .

Please check below sample code for your requirement.

For KV Store Lookup. Here, you can change the configuration per your requirement.

collections.conf

[kvstoreog] enforceTypes = true field.new_field = string

transforms.conf

[og_info] external_type = kvstore case_sensitive_match = true collection = kvstoreog fields_list = _key, new_field

Dashboard XML. Kindly change search and the key value as per your requirement.

<dashboard script="a.js"> <label>add free text to table</label> <row> <panel> <table id="tbl1"> <search> <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3 0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1 68a521188b984d2b84e41d35e4ceffa9,timestamp,value2 e646271ee11c4264ba4747519f1ba0e5,timestamp,value3 925fd4c2dc27413384f1a51c11721864,timestamp,value4 b8a9848134a243b684a44e4895da6c8f,timestamp,value5 jjj9848134a243b684a44e4895da6c8f,timestamp,value6 " | multikv forceheader=1 | eval a=10 | accum a | eval n=1648188616 | eval OG_Field_2 = n - a | table OG_Field_1 OG_Field_2 OG_Field_3 | rename comment as "Upto now is for sample data only" | eval New_Field = OG_Field_1."_".OG_Field_2</query> <earliest>-24h@h</earliest> <latest>now</latest> </search> <option name="drilldown">none</option> <option name="refresh.display">progressbar</option> </table> </panel> </row> <row> <panel> <table> <search id="search1"> <query>| makeresults | eval _raw="OG_Field_1,OG_Field_2,OG_Field_3 0b5377ee2a534a0db231afe84ee9c9b0,timestamp,value1 68a521188b984d2b84e41d35e4ceffa9,timestamp,value2 e646271ee11c4264ba4747519f1ba0e5,timestamp,value3 925fd4c2dc27413384f1a51c11721864,timestamp,value4 b8a9848134a243b684a44e4895da6c8f,timestamp,value5 jjj9848134a243b684a44e4895da6c8f,timestamp,value6 " | multikv forceheader=1 | eval a=10 | accum a | eval n=1648188616 | eval OG_Field_2 = n - a | table OG_Field_1 OG_Field_2 OG_Field_3 | rename comment as "Upto now is for sample data only" | eval data_key = OG_Field_1."_".OG_Field_2 | lookup og_info _key as data_key</query> <earliest>-24h@h</earliest> <latest>now</latest> </search> <option name="drilldown">none</option> <option name="refresh.display">progressbar</option> </table> </panel> </row> </dashboard>

a.js

require([ 'underscore', 'jquery', 'splunkjs/mvc', 'splunkjs/mvc/tableview', 'splunkjs/mvc/simplexml/ready!' ], function(_, $, mvc, TableView) { console.log("LAZY!!!"); var service = mvc.createService({ owner: "nobody" }); var mySearch = mvc.Components.get("search1"); // Add overlay buttons to table var CustomRangeRenderer = TableView.BaseCellRenderer.extend({ canRender: function(cell) { return _(["New_Field"]).contains(cell.field); }, render: function($td, cell) { if (cell.field === "New_Field") { var strHtmlInput = '<input type="text">'; $td.append(strHtmlInput).on("change", function(e) { console.log(e.target.value) console.log("Saved", (new Date).getTime()) // Provide Record to store var record = { "_key": cell.value, "new_field": e.target.value }; service.request("storage/collections/data/kvstoreog/" + cell.value, "GET", null, null, null, { "Content-Type": "application/json" }, null).done(function(response) { console.log(response); service.request("storage/collections/data/kvstoreog/" + cell.value, "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() { console.log("Update Done"); mySearch.startSearch(); }); }).error(function(err) { console.log(err.responseText); service.request("storage/collections/data/kvstoreog/", "POST", null, null, JSON.stringify(record), { "Content-Type": "application/json" }, null).done(function() { console.log("Insert Done"); mySearch.startSearch(); }); }); }); } } }); var sh = mvc.Components.get("tbl1"); if (typeof(sh) != "undefined") { sh.getVisualization(function(tableView) { // Add custom cell renderer and force re-render tableView.table.addCellRenderer(new CustomRangeRenderer()); tableView.table.render(); }); } });

Screen

To get batter understanding, please use same configuration and code for POC and do change to play with it.

Thanks
KV

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

Re: How to add free text to table?

oh_my_lawdy — Fri, 25 Mar 2022 16:55:47 GMT

@kamlesh_vaghela , thank you so much! This example is exactly what I needed

Re: How to add free text to table?

oh_my_lawdy — Fri, 25 Mar 2022 20:03:39 GMT

Hi @kamlesh_vaghela , I've been experimenting with the dashboard and script you provided, but my dashboard isn't allow for free text input and instead calculates the `New_Field` value as

OG_Field_1 + '_' + OG_Field_2

See screenshot

I've added the .conf files to $SPLUNK_HOME/etc/apps/search/local and a.js to $SPLUNK_HOME/etc/apps/search/appserver/static/ and then I created a new dashboard with your XML source code. I then restarted the server.

Is this the correct setup?

Re: How to add free text to table?

kamlesh_vaghela — Mon, 28 Mar 2022 05:22:00 GMT

@oh_my_lawdy

May be we need to bump the javascript.. Just open this link and click on bump button.

https://localhost:8000/en-US/_bump

Open dashboard and right click on dashboard- open Inspect Element and check any error in console?

If you found any error please share to us.

Re: How to add free text to table?

oh_my_lawdy — Mon, 28 Mar 2022 12:11:11 GMT

That fixed it, thanks!