topic Re: Command 'fields' is rewriting the column data in Knowledge Management

Why is command 'fields' rewriting the column data?

Shubhanker99 — Mon, 15 May 2023 14:02:15 GMT

Hello,

I am facing an issue with the SPL of a dashboard panel. If you see the 2 figures, the SPL above the last 2-3 lines is same. When doing 'fields -' and removing the unnecessary fields, the correct data is being outputted. But when doing 'fields' to take just the required fields and then removing '_raw' with 'fields -', the values are being overwritten? Note that the field of 'workflow_username' has no issues and only 'totalScore' and 'percentage' fields are having this issue.

Another thing to note that the 'totalScore' field is derived from other data using 'foreach' and 'eval' commands but I don't think this issue is because of that.

Any help is appreciated. Thanks.

Figure 1

Figure 2

Re: Command 'fields' is rewriting the column data

yeahnah — Mon, 15 May 2023 04:48:04 GMT

Hi@Shubhanker99

Yes, that is strange. If you provide all your SPL then that might help understand what the issue is.

Or, in the case, as you only want to display 4 columns, just use the table command as this removes any _* columns by default, if not otherwise specified, e.g.

...
| table ts totalScore percentage workflow_username

Hope this helps

Re: Command 'fields' is rewriting the column data

Shubhanker99 — Mon, 15 May 2023 05:17:34 GMT

@yeahnah here is the edited SPL:

`setPrimaryIndex` sourcetype="A" | search type=B | rename workflow_step.inputs.input.value as _raw | extract pairdelim="|", kvdelim="=" | rename _raw as workflow_step.inputs.input.value | stats min(event_ts) as ts values(workflow_step.inputs.workflow_name_input.value) as workflow_name values(workflow_step.inputs.total_questions_input.value) as total_questions values(workflow_step.inputs.input.value) as text values(workflow_step.inputs.workflow_executor_input.value) as workflow_executor by workflow_step.workflow_instance_id | where workflow_name="C" | rename text as _raw | extract pairdelim="|", kvdelim="=" | rename _raw as text | fields - workflow_step.workflow_instance_id status | join type=outer workflow_executor [|`getChatUserLatest` | rename id as workflow_executor, name as workflow_username | eval workflow_executor="<@". workflow_executor . ">" | fields - _*| fields workflow_executor workflow_username] | fields - workflow_executor text | sort - ts | eval ts=strftime(ts,"%F %T") | rename workflow_name as workflow | lookup D workflow OUTPUTNEW answer_sheet | rename answer_sheet as _raw | extract kvdelim="=" pairdelim="," | eval totalScore = 0 | foreach q* [eval totalScore=if(lower(q<<MATCHSTR>>)=lower(a<<MATCHSTR>>), totalScore+1, totalScore)] | search q1 = * | dedup workflow_username sortby - _time | eval percentage = round(totalScore/total_questions * 100) | eval percentage = percentage + "%" | fields - _raw total_questions workflow a* q* | sort - totalScore

As you said using 'table' will work but I want to know why 'field' command is behaving in this way.

Re: Command 'fields' is rewriting the column data

yeahnah — Mon, 15 May 2023 21:20:11 GMT

Thanks @Shubhanker99

Nothing obviously wrong in the SPL. Could be a bug so next steps would be raising a support case with Splunk.

Re: Command 'fields' is rewriting the column data

Shubhanker99 — Tue, 16 May 2023 01:17:39 GMT

Thanks @yeahnah will do that.