Getting Data In

how to export csv with BOM ?

newsm106
Explorer

Hi,

I have a csv file encoded UTF-8 from Scheduled Search. And MS Excel couldn't read the file Because without BOM I guess. How can I export csv file with BOM ??

I searched on Splunk and found this.
http://answers.splunk.com/answers/5610/can-the-export-function-export-csv-file-in-big5-charset

In the file
$SPLUNK_HOME/lib/python2.6/site-packages/splunk/appserver/mrsparkle/controllers/search.py
go to around line 266, where the
statement output = job.getFeed(asset,
http_method=jobFeedRequestMethod,
**kwargs) is.

Add the following code after that
line:

if 'isDownload' in kwargs:
    import codecs
    output = "".join((codecs.BOM_UTF8, output))

This should allow the files to be read
by Excel.

But My Python version is 2.7. So the code of Search.py is different.

How can I get CSV with BOM??

Regards,

Sam

Tags (4)

jeffland
SplunkTrust
SplunkTrust

For anyone interested in a solution for recent Splunk versions, go to the file

splunkhome/lib/python2.7/site-packages/splunk/rest/__init__.py

to line 648 where it says

def readall(self, blocksize=32768):
    """
    Returns a generator reading blocks of data from the response
    until all data has been read
    """
    response = self.response
    while 1:
        data = response.read(blocksize)
        if not data:
            break
        yield data

Change that to

def readall(self, blocksize=32768):
    """
    Returns a generator reading blocks of data from the response
    until all data has been read
    """
    response = self.response
    import codecs
    counter = 0;
    while 1:
        data = response.read(blocksize)
        if not data:
            break

        if counter == 0:
            data = "".join((codecs.BOM_UTF8, data))
            counter += 1

        yield data

and your exports come with BOM.

ktc78
Explorer

jeffland's code works perfectly for previous splunk version

but in case of splunk v8.0.0,
I'm afraid python 3.7 is used

splunkhome/lib/python3.7/site-packages/splunk/rest/__init__.py

and raised error like below

Unrecoverable error in the server.
Traceback (most recent call last):
File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpwsgi.py", line 184, in trap
return func(*args, **kwargs)
File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpwsgi.py", line 277, in __next__
return next(self.iter_response)
File "/opt/splunk/lib/python3.7/site-packages/cherrypy/lib/encoding.py", line 99, in encoder
for chunk in body:
File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 716, in readall
data = "".join((codecs.BOM_UTF8, data))
TypeError: sequence item 0: expected str instance, bytes found

Can I have code for python3.7?

Best regards,
Kang

kichonei
Engager

hey, Kang

[python 2.7]
data = "".join((codecs.BOM_UTF8, data))

[python 3.7]
: Add 'b' in front.

data = b"".join((codecs.BOM_UTF8, data))

SwatiApte
Path Finder

Hi,

When I added the above mentioned code in Python 2.7 search.py, and restarted splunk, the Web service did not start at all. The command prompt was stuck at "Waiting for the web service to be available". As soon as I removed the added three lines, it started working. Could you please let me know if there is any other way of doing the same, or whether the code is placed in the correct place.

0 Karma

mcseem
Explorer

Tried to append code after line 342 (output = job_lite.get(asset))
No result. Files still exporting without BOM

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...