splunk result has fields unwanted

[Anthony_Hou]

Dear All,

we have encountered one problem

we designed a script to find out where the result is.

20110112_182817 result_fl = /opt/splunk/var/run/splunk/dispatch/scheduler_owenchuang_search_Q1BFLUFTQ0NfU05NUC1UUkFQLUxJTkstRE9XTi10ZXN0XzExMV8x_at_1294828080_948519812/results.csv.gz

The result is as below but there are some fields beginning with "_mv"

"Date_time_2",NeName,"family_type_detail",iFName,"__mv_Date_time_2","__mv_NeName","__mv_family_type_detail","__mv_iFName" "Jan 12 00:58:50","ASCC-HKBR0: 202.169.111.249","SNMP_TRAP_LINK_DOWN","so-3/0/0.6",,,, "Jan 12 00:58:50","ASCC-HKBR0: 202.169.111.249","SNMP_TRAP_LINK_DOWN","so-3/0/0.0",,,, "Jan 12 00:58:50","ASCC-JPBR0: 202.169.111.250","SNMP_TRAP_LINK_DOWN","so-4/2/0",,,,

We have tried "* | fields -_" and also " | fields - _mv*" but it's not working

if we ask splunk to just show the fields we want, and ignore the fields we don't want, why are some fields still coming out?

Is there any solution that we can remove the fields we do not want?

then we can put the correct result to our alarm system.

tks for any suggestions.

Anthony

[dmlee]

please refer below shell command :

cat $original_Filename | sed 's/,"__mv[^,]*//g' | sed 's/,"_*"//g' | sed 's/,,*/,/g' | sed 's/,$//g' > $your_Filename

[mw]

That won't remove the values/rows associated with those __mv, etc headers.

[steveyz]

the results.csv.gz file in each job's directory was not intended to be directly consumed by users. The __mv_* fields are special encoded fields used to represent multi-value fields. If you want the results of a search in the form of a csv file, I'd suggest using the outputcsv command at the end of your search, which will write the output to $SPLUNK_HOME/var/run/splunk/ directory (instead of the job's own directory).

Another workaround is to use a script to strip any fields that start with __ (2 leading underscores denote a purely special internal field) before passing into your alarm system.

[mw]

If you create an alert, one of the arguments passed to your script will be the the path to the search results. This is useful for integrating with other systems, and makes me wonder why it wouldn't be expected that you use those results.

[KimiYan]

How old is this question... Currently I have the same problem with you. Have you solved it finally ?

[steveyz]

Have you tried just using the REST api to retrive the results? if you just point to

https://:/services/search/jobs//results?output_mode=csv

You should get the rests in csv format, which you can certainly save to a file.

[Anthony_Hou]

Tks a lot, Steveyz. tks for your quickly answer.
we have tried to filter some words, like begin with __mv
but it is hard to avoid words that we still do not know in future.
If the result is not intended for users, why splunk tell us where is the result?
If we use outputcsv, do we have to design different csv filename for every savedsearch? There might be possible that csv file will be overwrited. I think that is why Splunk uses random directory to store the result. splunk tell us where the unique result is, then not wish us to use it. That is a little strange and unreasonable. ^^