set encoding UTF8-BOM

[silverKi]

I want to use the splunk app for lookup file editing to export a csv lookup file and automatically apply utf8-bom encoding when opening the downloaded file on my local PC. How can I do this?

I went into that path and modified lookup_editor_rest_handler.py and added a utf8-BOM stream to csv_data, but that setting is not applied. When the user opens the exported Excel file on a local PC, a problem occurs where non-English characters are encoded incorrectly.

$SPLUNK_HOME/etc/apps/lookup_editor/bin/lookup_editor_rest_handler.py

import codecs


def post_lookup_as_file(self, request_info, lookup_file=None, namespace="lookup_editor",
                                owner=None, lookup_type='csv', **kwargs):
    
             self.logger.info("Exporting lookup, namespace=%s, lookup=%s, type=%s, owner=%s", namespace,
                             lookup_file, lookup_type, owner)
            try:
    
                 # If we are getting the CSV, then just pipe the file to the user
                if lookup_type == "csv":
                     with self.lookup_editor.get_lookup(request_info.session_key, lookup_file, namespace, owner) as csv_file_handle:
                         csv_data = csv_file_handle.read()
                         csv_data = codecs.BOM_UTF8.decode('utf-8')+csv_data
                      
    
                 # If we are getting a KV store lookup, then convert it to a CSV file
                 else:
                     rows = self.lookup_editor.get_kv_lookup(request_info.session_key, lookup_file, namespace, owner)
                     csv_data = shortcuts.convert_array_to_csv(rows)
                     
                 return {
                         'payload': csv_data, # Payload of the request.
                         'status': 200, # HTTP status code
                         'headers': {
                         'Content-Type': 'text/csv; charset=UTF-8',
                         'Content-Disposition': f'attachment; filename*=UTF-8\'\'{lookup_file}'
                     },
    
                     }
             except (IOError, ResourceNotFound):
                 return self.render_error_json("Unable to find the lookup", 404)
    
             except (AuthorizationFailed, PermissionDeniedException):
                return self.render_error_json("You do not have permission to perform this operation", 403)
    
             except Exception as e:
                 self.logger.exception("Export lookup: details=%s", e)
                 return self.render_error_json("Something went wrong!")

[PickleRick]

1. BOM just adds three bytes at the beginning of the file. That's it. It doesn't change anything within the utf-8 encoding.

2. Did you verify that those bytes are added on export?

3. Did you verify how those bytes are exported? (see the raw file contents and check the actual code points)

4. Excel isn't always right, you know?

[silverKi]

I want to  convert UTF-8 text to UTF8 with BOM text