Getting Data In

how to export csv with BOM ?

newsm106
Explorer

Hi,

I have a csv file encoded UTF-8 from Scheduled Search. And MS Excel couldn't read the file Because without BOM I guess. How can I export csv file with BOM ??

I searched on Splunk and found this.
http://answers.splunk.com/answers/5610/can-the-export-function-export-csv-file-in-big5-charset

In the file
$SPLUNK_HOME/lib/python2.6/site-packages/splunk/appserver/mrsparkle/controllers/search.py
go to around line 266, where the
statement output = job.getFeed(asset,
http_method=jobFeedRequestMethod,
**kwargs) is.

Add the following code after that
line:

if 'isDownload' in kwargs:
    import codecs
    output = "".join((codecs.BOM_UTF8, output))

This should allow the files to be read
by Excel.

But My Python version is 2.7. So the code of Search.py is different.

How can I get CSV with BOM??

Regards,

Sam

Tags (4)

jeffland
SplunkTrust
SplunkTrust

For anyone interested in a solution for recent Splunk versions, go to the file

splunkhome/lib/python2.7/site-packages/splunk/rest/__init__.py

to line 648 where it says

def readall(self, blocksize=32768):
    """
    Returns a generator reading blocks of data from the response
    until all data has been read
    """
    response = self.response
    while 1:
        data = response.read(blocksize)
        if not data:
            break
        yield data

Change that to

def readall(self, blocksize=32768):
    """
    Returns a generator reading blocks of data from the response
    until all data has been read
    """
    response = self.response
    import codecs
    counter = 0;
    while 1:
        data = response.read(blocksize)
        if not data:
            break

        if counter == 0:
            data = "".join((codecs.BOM_UTF8, data))
            counter += 1

        yield data

and your exports come with BOM.

ktc78
Explorer

jeffland's code works perfectly for previous splunk version

but in case of splunk v8.0.0,
I'm afraid python 3.7 is used

splunkhome/lib/python3.7/site-packages/splunk/rest/__init__.py

and raised error like below

Unrecoverable error in the server.
Traceback (most recent call last):
File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpwsgi.py", line 184, in trap
return func(*args, **kwargs)
File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpwsgi.py", line 277, in __next__
return next(self.iter_response)
File "/opt/splunk/lib/python3.7/site-packages/cherrypy/lib/encoding.py", line 99, in encoder
for chunk in body:
File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 716, in readall
data = "".join((codecs.BOM_UTF8, data))
TypeError: sequence item 0: expected str instance, bytes found

Can I have code for python3.7?

Best regards,
Kang

kichonei
Engager

hey, Kang

[python 2.7]
data = "".join((codecs.BOM_UTF8, data))

[python 3.7]
: Add 'b' in front.

data = b"".join((codecs.BOM_UTF8, data))

SwatiApte
Path Finder

Hi,

When I added the above mentioned code in Python 2.7 search.py, and restarted splunk, the Web service did not start at all. The command prompt was stuck at "Waiting for the web service to be available". As soon as I removed the added three lines, it started working. Could you please let me know if there is any other way of doing the same, or whether the code is placed in the correct place.

0 Karma

mcseem
Explorer

Tried to append code after line 342 (output = job_lite.get(asset))
No result. Files still exporting without BOM

0 Karma
Get Updates on the Splunk Community!

Thanks for the Memories! Splunk University, .conf24, and Community Connections

Thank you to everyone in the Splunk Community who joined us for .conf24 – starting with Splunk University and ...

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...