Splunk Dev

Splunk python with pandas

to4kawa
Ultra Champion

Splunk python3 can't work import pandas
How should I do?

sample:

% $SPLUNK_HOME/bin/splunk cmd python3
>>> import pandas
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'

and how does custom apps use python3?
where should I put on python.version=python3?

Labels (1)
0 Karma
1 Solution

seva98
Path Finder

Try manually downloading pandas and copy them to Splunk\etc\apps\<APP>\bin folder then in script that you have in that bin folder, for example Splunk\etc\apps\<APP>\bin\main.py add following:

import sys
sys.path.append('./pandas')
sys.path.insert(0, './pandas')

It is lazy workaround but importing should now work.

View solution in original post

stevejonn
Observer

In most cases this error in Python generally raised:

  • You haven't installed Pandas explicitly with pip install pandas.
  • You may have different Python versions on your computer and Pandas is not installed for the particular version you're using.

You can run the following command in your Linux/MacOS/Windows terminal.

pip install pandas

To be sure you are not having multiple Python versions that are confusing, you should run following commands:

python3 -m pip install pandas
python3 -c 'import pandas'

 

0 Karma

seva98
Path Finder

Try manually downloading pandas and copy them to Splunk\etc\apps\<APP>\bin folder then in script that you have in that bin folder, for example Splunk\etc\apps\<APP>\bin\main.py add following:

import sys
sys.path.append('./pandas')
sys.path.insert(0, './pandas')

It is lazy workaround but importing should now work.

sarvesh_11
Communicator

Hello,

I tried this work around for pandas.

but i am getting the below error while calling pandas.

Traceback (most recent call last):
File "sample.py", line 38, in <module>
df = pandas.DataFrame(data)
NameError: name 'pandas' is not defined

while i added syspath append and insert, i removed import pandas from the python file.

0 Karma

to4kawa
Ultra Champion

@seva98
thank you. I can't find this solution.

0 Karma

to4kawa
Ultra Champion
0 Karma

to4kawa
Ultra Champion
0 Karma

PavelP
Motivator

Hello @to4kawa,

I think the pandas module wasn't properly installed. Can you list your steps and reference to the documentation you've used to install pandas python module?

0 Karma

to4kawa
Ultra Champion

hi @PaveIP

!#/usr/bin/env python

It run with python in my OS(e.g. Anaconda) . I can use pandas ,also.

I'm trying to publish Apps, and considering windows, if pandas can be used with Splunk's standard python, I'm trying to change to that method like following:

!#$SPLUNK_HOME/bin/python3

However, I have not found it yet. Should I copy Python for Scientific Computing/exec_anaconda.py ?

reference: https://github.com/splunk/Splunk-python-for-scientific-computing/tree/master/package

0 Karma

PavelP
Motivator

I'm not sure, that can work

0 Karma

to4kawa
Ultra Champion

Thanks @PaveIP
No problem.

0 Karma

to4kawa
Ultra Champion

sample usage:

transforms.conf

[freqserver]
external_cmd = freq.py domain
external_type = python
python.version = python2
fields_list = domain, frequency

ref: https://answers.splunk.com/answering/816676/view.html

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...