How to upload a .csv file onto a Splunk remote server using Python?

[vkannampuzha]

Hi all,

I am trying to upload a .csv file onto a remote Splunk server through the use of a Python script and I am having a bit of difficulty in getting this to run. The methods I have tried are:

Method 1: Creating a service and then connecting to the server

 """Connect to splunk local"""

     import splunklib.client as client
     import splunklib.results as results
     from splunklib.binding import AuthenticationError

     HOST=hostname
     PORT = '8089'
     USERNAME = 'username'
     PASSWORD = 'password'
     try:
         service = client.connect(host=HOST, port=PORT, username=USERNAME, password=PASSWORD)
     except exception as e:
         print(str(e))

The issue with this was that it never seemed to connect. Please note that the host was not a local host but rather a remote Splunk server.

Method 2: Using the HTTP Event Collector

Whilst the HTTP event collector worked well in sending the data through line by line, however, the data needs to be in csv format, not JSON, which it is converted to.

Any thoughts and suggestions on how to proceed would be much helpful! Would prefer to be python based solution rather than setting up a file monitor on Splunk

Cheers,

[starcher · Accepted solution]

If you mean csv lookup ,csv lookups are not part of the REST API. You'd need to use KVStore. if this is data why does it need to be csv? JSON with HEC is substantially simpler. If it absolutely must be csv data vs a lookup you can always use RAW mode with HEC.

[starcher]

https://github.com/georgestarcher/Splunk-Class-httpevent

[vkannampuzha]

I do agree with JSON being much simpler with HEC, however, the data is being used for pre-existing models that require the format to be csv. I just tried the RAW mode and that seems to be much better. Is there any way for Splunk to recognise fields then with this method?

Thanks very much for your help!

[starcher]

Read the docs on getting data into Splunk. Setup your sourcetype. setup it's parsing etc.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Getstartedwithgettingdatain