Getting Data In

Using pandas (in PSC) inside a Python Modular Input

ownion
Path Finder

Hello Everyone,

I'm trying to write a custom Python Modular Input to fetch some HTML tables (all the Windows 10 release history tables) from the Microsoft Windows 10 Release Information.

My idea is to create a modular input that runs once a month and uses pandas.read_html function to ingest all the Release History Tables and index all the rows into Splunk.

I've figured out how to do the Python code but I've some issues with importing the pandas library into my custom app, I've read some Splunk Community posts and I've placed the exec_anaconda.py (from $SPLUNK_HOME\etc\apps\Splunk_SA_Scientific_Python_windows_x86_64\bin) inside %SPLUNK_HOME%\etc\apps\my_custom_app\bin and  also added the util folder (from %SPLUNK_HOME%\etc\apps\Splunk_ML_Toolkit\bin\utils) to avoid  the "ModuleNotFoundError: No module named 'util'" Python Exception.

Also, as stated in the PSC README, I've placed the following lines right under the beginning of def collect_events(...) function:

 

 

def collect_event(helper, ew):
    import exec_anaconda
    exec_anaconda.exec_anaconda()
    import pandas
    ...

 

 

I keep getting the error: "ERROR Error encountered while loading Python for Scientific Computing, see search.log." But obviusly the search.log file is empty since this is not a SPL search.

 

Is it possible to use the PSC libraries inside my modular input to accomplish this?

Thank you.

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...