topic Re: Splunk python with pandas in Splunk Dev

Splunk python with pandas

to4kawa — Sun, 07 Jun 2020 18:14:45 GMT

Splunk python3 can't work import pandas
How should I do?

sample:

% $SPLUNK_HOME/bin/splunk cmd python3
>>> import pandas
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'

and how does custom apps use python3?
where should I put on python.version=python3?

Re: Splunk python with pandas

to4kawa — Sun, 12 Apr 2020 01:55:09 GMT

sample usage:

transforms.conf

[freqserver]
external_cmd = freq.py domain
external_type = python
python.version = python2
fields_list = domain, frequency

ref: https://answers.splunk.com/answering/816676/view.html

Re: Splunk python with pandas

to4kawa — Sun, 12 Apr 2020 01:57:47 GMT

reference:https://docs.splunk.com/Documentation/Splunk/latest/Python3Migration/ChangesEnterprise#Python_interpreter_settings

Re: Splunk python with pandas

to4kawa — Mon, 13 Apr 2020 09:45:41 GMT

https://dev.splunk.com/enterprise/docs/developapps/customsearchcommands/createcustomsearchcmd/

Re: Splunk python with pandas

to4kawa — Mon, 13 Apr 2020 09:47:13 GMT

https://stackoverflow.com/questions/55275491/splunk-splunk-python-sdk-how-to-include-pandas-numpy-to-create-custom-command

Re: Splunk python with pandas

to4kawa — Mon, 13 Apr 2020 09:53:44 GMT

https://answers.splunk.com/answers/681860/python-for-scientific-computing-app-error-when-imp.html

Re: Splunk python with pandas

PavelP — Mon, 13 Apr 2020 22:07:44 GMT

Hello @to4kawa,

I think the pandas module wasn't properly installed. Can you list your steps and reference to the documentation you've used to install pandas python module?

Re: Splunk python with pandas

to4kawa — Wed, 30 Sep 2020 05:00:29 GMT

hi @PaveIP

!#/usr/bin/env python

It run with python in my OS(e.g. Anaconda) . I can use pandas ,also.

I'm trying to publish Apps, and considering windows, if pandas can be used with Splunk's standard python, I'm trying to change to that method like following:

!#$SPLUNK_HOME/bin/python3

However, I have not found it yet. Should I copy Python for Scientific Computing/exec_anaconda.py ?

reference: https://github.com/splunk/Splunk-python-for-scientific-computing/tree/master/package

Re: Splunk python with pandas

PavelP — Tue, 14 Apr 2020 06:11:46 GMT

I'm not sure, that can work

Re: Splunk python with pandas

to4kawa — Tue, 14 Apr 2020 06:32:17 GMT

Thanks @PaveIP
No problem.

Re: Splunk python with pandas

to4kawa — Wed, 15 Apr 2020 21:27:17 GMT

https://answers.splunk.com/answers/795799/how-to-import-util-when-using-exec-anaconda.html

Re: Splunk python with pandas

to4kawa — Wed, 15 Apr 2020 21:27:50 GMT

https://answers.splunk.com/answers/686808/splunk-machine-learning-toolkit-install-additional.html

Re: Splunk python with pandas

to4kawa — Wed, 15 Apr 2020 21:39:27 GMT

https://static.rainfocus.com/splunk/splunkconf18/sess/1523381218451001Aj1y/finalPDF/FN1409_GitHubCollaboration_Final_1538799477337001rcTB.pdf

Re: Splunk python with pandas

to4kawa — Mon, 20 Apr 2020 10:30:09 GMT

https://docs.splunk.com/Documentation/MLApp/latest/API/AddPython3Libs

Re: Splunk python with pandas

to4kawa — Mon, 20 Apr 2020 10:54:11 GMT

https://github.com/splunk/mltk-algo-contrib

Re: Splunk python with pandas

to4kawa — Fri, 24 Apr 2020 06:51:21 GMT

https://docs.splunk.com/Documentation/Splunk/8.0.3/AdvancedDev/ScriptWriting

Re: Splunk python with pandas

seva98 — Wed, 29 Apr 2020 09:17:30 GMT

Try manually downloading pandas and copy them to Splunk\etc\apps\<APP>\bin folder then in script that you have in that bin folder, for example Splunk\etc\apps\<APP>\bin\main.py add following:

import sys
sys.path.append('./pandas')
sys.path.insert(0, './pandas')

It is lazy workaround but importing should now work.

Re: Splunk python with pandas

to4kawa — Wed, 29 Apr 2020 09:39:00 GMT

@seva98
thank you. I can't find this solution.

Re: Splunk python with pandas

stevejonn — Mon, 12 Apr 2021 05:45:46 GMT

In most cases this error in Python generally raised:

You haven't installed Pandas explicitly with pip install pandas.
You may have different Python versions on your computer and Pandas is not installed for the particular version you're using.

You can run the following command in your Linux/MacOS/Windows terminal.

pip install pandas

To be sure you are not having multiple Python versions that are confusing, you should run following commands:

python3 -m pip install pandas
python3 -c 'import pandas'

Re: Splunk python with pandas

sarvesh_11 — Wed, 22 Dec 2021 06:34:58 GMT

Hello,

I tried this work around for pandas.

but i am getting the below error while calling pandas.

Traceback (most recent call last):
File "sample.py", line 38, in <module>
df = pandas.DataFrame(data)
NameError: name 'pandas' is not defined

while i added syspath append and insert, i removed import pandas from the python file.