Splunk Search

splunk result has fields unwanted

Anthony_Hou
Path Finder

Dear All,

we have encountered one problem

we designed a script to find out where the result is.

20110112_182817 result_fl = /opt/splunk/var/run/splunk/dispatch/scheduler_owenchuang_search_Q1BFLUFTQ0NfU05NUC1UUkFQLUxJTkstRE9XTi10ZXN0XzExMV8x_at_1294828080_948519812/results.csv.gz

The result is as below but there are some fields beginning with "_mv"

"Date_time_2",NeName,"family_type_detail",iFName,"__mv_Date_time_2","__mv_NeName","__mv_family_type_detail","__mv_iFName" "Jan 12 00:58:50","ASCC-HKBR0: 202.169.111.249","SNMP_TRAP_LINK_DOWN","so-3/0/0.6",,,, "Jan 12 00:58:50","ASCC-HKBR0: 202.169.111.249","SNMP_TRAP_LINK_DOWN","so-3/0/0.0",,,, "Jan 12 00:58:50","ASCC-JPBR0: 202.169.111.250","SNMP_TRAP_LINK_DOWN","so-4/2/0",,,,

We have tried "* | fields -_" and also " | fields - _mv*" but it's not working

if we ask splunk to just show the fields we want, and ignore the fields we don't want, why are some fields still coming out?

Is there any solution that we can remove the fields we do not want?

then we can put the correct result to our alarm system.

tks for any suggestions.

Anthony

Tags (2)

dmlee
Communicator

please refer below shell command :

cat $original_Filename | sed 's/,"__mv[^,]*//g' | sed 's/,"_*"//g' | sed 's/,,*/,/g' | sed 's/,$//g' > $your_Filename

0 Karma

mw
Splunk Employee
Splunk Employee

That won't remove the values/rows associated with those __mv, etc headers.

steveyz
Splunk Employee
Splunk Employee

the results.csv.gz file in each job's directory was not intended to be directly consumed by users. The __mv_* fields are special encoded fields used to represent multi-value fields. If you want the results of a search in the form of a csv file, I'd suggest using the outputcsv command at the end of your search, which will write the output to $SPLUNK_HOME/var/run/splunk/ directory (instead of the job's own directory).

Another workaround is to use a script to strip any fields that start with __ (2 leading underscores denote a purely special internal field) before passing into your alarm system.

mw
Splunk Employee
Splunk Employee

If you create an alert, one of the arguments passed to your script will be the the path to the search results. This is useful for integrating with other systems, and makes me wonder why it wouldn't be expected that you use those results.

KimiYan
New Member

How old is this question... Currently I have the same problem with you. Have you solved it finally ?

0 Karma

steveyz
Splunk Employee
Splunk Employee

Have you tried just using the REST api to retrive the results? if you just point to

https://:/services/search/jobs//results?output_mode=csv

You should get the rests in csv format, which you can certainly save to a file.

0 Karma

Anthony_Hou
Path Finder

Tks a lot, Steveyz. tks for your quickly answer.
we have tried to filter some words, like begin with __mv
but it is hard to avoid words that we still do not know in future.
If the result is not intended for users, why splunk tell us where is the result?
If we use outputcsv, do we have to design different csv filename for every savedsearch? There might be possible that csv file will be overwrited. I think that is why Splunk uses random directory to store the result. splunk tell us where the unique result is, then not wish us to use it. That is a little strange and unreasonable. ^^

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...