Building for the Splunk Platform

Splunk python with pandas

to4kawa
Ultra Champion

Splunk python3 can't work import pandas
How should I do?

sample:

% $SPLUNK_HOME/bin/splunk cmd python3
>>> import pandas
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'

and how does custom apps use python3?
where should I put on python.version=python3?

Labels (1)
0 Karma
1 Solution

seva98
Path Finder

Try manually downloading pandas and copy them to Splunk\etc\apps\<APP>\bin folder then in script that you have in that bin folder, for example Splunk\etc\apps\<APP>\bin\main.py add following:

import sys
sys.path.append('./pandas')
sys.path.insert(0, './pandas')

It is lazy workaround but importing should now work.

View solution in original post

stevejonn
Observer

In most cases this error in Python generally raised:

  • You haven't installed Pandas explicitly with pip install pandas.
  • You may have different Python versions on your computer and Pandas is not installed for the particular version you're using.

You can run the following command in your Linux/MacOS/Windows terminal.

pip install pandas

To be sure you are not having multiple Python versions that are confusing, you should run following commands:

python3 -m pip install pandas
python3 -c 'import pandas'

 

0 Karma

seva98
Path Finder

Try manually downloading pandas and copy them to Splunk\etc\apps\<APP>\bin folder then in script that you have in that bin folder, for example Splunk\etc\apps\<APP>\bin\main.py add following:

import sys
sys.path.append('./pandas')
sys.path.insert(0, './pandas')

It is lazy workaround but importing should now work.

sarvesh_11
Communicator

Hello,

I tried this work around for pandas.

but i am getting the below error while calling pandas.

Traceback (most recent call last):
File "sample.py", line 38, in <module>
df = pandas.DataFrame(data)
NameError: name 'pandas' is not defined

while i added syspath append and insert, i removed import pandas from the python file.

0 Karma

to4kawa
Ultra Champion

@seva98
thank you. I can't find this solution.

0 Karma

to4kawa
Ultra Champion
0 Karma

to4kawa
Ultra Champion
0 Karma

PavelP
Motivator

Hello @to4kawa,

I think the pandas module wasn't properly installed. Can you list your steps and reference to the documentation you've used to install pandas python module?

0 Karma

to4kawa
Ultra Champion

hi @PaveIP

!#/usr/bin/env python

It run with python in my OS(e.g. Anaconda) . I can use pandas ,also.

I'm trying to publish Apps, and considering windows, if pandas can be used with Splunk's standard python, I'm trying to change to that method like following:

!#$SPLUNK_HOME/bin/python3

However, I have not found it yet. Should I copy Python for Scientific Computing/exec_anaconda.py ?

reference: https://github.com/splunk/Splunk-python-for-scientific-computing/tree/master/package

0 Karma

PavelP
Motivator

I'm not sure, that can work

0 Karma

to4kawa
Ultra Champion

Thanks @PaveIP
No problem.

0 Karma

to4kawa
Ultra Champion

sample usage:

transforms.conf

[freqserver]
external_cmd = freq.py domain
external_type = python
python.version = python2
fields_list = domain, frequency

ref: https://answers.splunk.com/answering/816676/view.html

0 Karma
Get Updates on the Splunk Community!

New Splunk Observability innovations: Deeper visibility and smarter alerting to ...

You asked, we delivered. Splunk Observability Cloud has several new innovations giving you deeper visibility ...

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...