Getting Data In

How to upload a .csv file onto a Splunk remote server using Python?

vkannampuzha
Explorer

Hi all,

I am trying to upload a .csv file onto a remote Splunk server through the use of a Python script and I am having a bit of difficulty in getting this to run. The methods I have tried are:

Method 1: Creating a service and then connecting to the server

 """Connect to splunk local"""

     import splunklib.client as client
     import splunklib.results as results
     from splunklib.binding import AuthenticationError

     HOST=hostname
     PORT = '8089'
     USERNAME = 'username'
     PASSWORD = 'password'
     try:
         service = client.connect(host=HOST, port=PORT, username=USERNAME, password=PASSWORD)
     except exception as e:
         print(str(e))

The issue with this was that it never seemed to connect. Please note that the host was not a local host but rather a remote Splunk server.

Method 2: Using the HTTP Event Collector

Whilst the HTTP event collector worked well in sending the data through line by line, however, the data needs to be in csv format, not JSON, which it is converted to.

Any thoughts and suggestions on how to proceed would be much helpful! Would prefer to be python based solution rather than setting up a file monitor on Splunk

Cheers,

Tags (1)
0 Karma
1 Solution

starcher
SplunkTrust
SplunkTrust

If you mean csv lookup ,csv lookups are not part of the REST API. You'd need to use KVStore. if this is data why does it need to be csv? JSON with HEC is substantially simpler. If it absolutely must be csv data vs a lookup you can always use RAW mode with HEC.

View solution in original post

0 Karma

starcher
SplunkTrust
SplunkTrust

If you mean csv lookup ,csv lookups are not part of the REST API. You'd need to use KVStore. if this is data why does it need to be csv? JSON with HEC is substantially simpler. If it absolutely must be csv data vs a lookup you can always use RAW mode with HEC.

0 Karma

starcher
SplunkTrust
SplunkTrust
0 Karma

vkannampuzha
Explorer

I do agree with JSON being much simpler with HEC, however, the data is being used for pre-existing models that require the format to be csv. I just tried the RAW mode and that seems to be much better. Is there any way for Splunk to recognise fields then with this method?

Thanks very much for your help!

0 Karma

starcher
SplunkTrust
SplunkTrust

Read the docs on getting data into Splunk. Setup your sourcetype. setup it's parsing etc.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Getstartedwithgettingdatain

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...