Hello! We currently index a variety of XML files that we auto-extract the fields from using XML KV. Because the XMLs are a flattened version of system files with a variety of classes/loops, the fields end up being named something like "explanation_of_benefits.member.address.zip" or something along those lines.
We created a suite of aliases to give them friendlier names when searching, but the original fields are still present, clogging up the "interesting fields" space if you search in verbose/smart mode.
Is there any way to hide these original fields? I found some old posts saying it might have been possible in v5, but once v6 came out that solution stopped working.
Thanks!
Long time ago now but if anyone else comes looking here's a solution
Best thing to do is in props if you're using KV_MODE=XML
take that off, if you're not using KV, no worries. Then EVAL<field_name> = spath
all the fields you want to keep
EG:
EVAL-ZIP = spath(_raw,'explanation_of_benefits.member.address.zip')
This will give you a field called ZIP with the value you need and no full path.
Also if you still need to get back to the full paths temporally for any reason (looking at a new field) you can just use spath again in the search head:
index=explain sourcetype=benifits | spath
This will give you all the paths in the _raw
Hope this helps someone.
the only way I got around this issue is by moving my data into another index:
after I ran this "all time" I scheduled a report with the search below, to update my "clean" index every hour for the last hour.
index=myxml sourcetype=toomanyfields
| table clean_field1 clean_field2 clean_field3
| collect index=clean sourcetype=clean_data
Have you tried something like ... | fields - explanation_of_benefits.* | ...
?
Hi Rich,
Unfortunately we were hoping for something that we wouldn't have to add to each search, as there are about 20-30 of these fields for each file type. But thank you for your response!