Using pandas (in PSC) inside a Python Modular Input

[ownion]

Hello Everyone,

I'm trying to write a custom Python Modular Input to fetch some HTML tables (all the Windows 10 release history tables) from the Microsoft Windows 10 Release Information.

My idea is to create a modular input that runs once a month and uses pandas.read_html function to ingest all the Release History Tables and index all the rows into Splunk.

I've figured out how to do the Python code but I've some issues with importing the pandas library into my custom app, I've read some Splunk Community posts and I've placed the exec_anaconda.py (from $SPLUNK_HOME\etc\apps\Splunk_SA_Scientific_Python_windows_x86_64\bin) inside %SPLUNK_HOME%\etc\apps\my_custom_app\bin and  also added the util folder (from %SPLUNK_HOME%\etc\apps\Splunk_ML_Toolkit\bin\utils) to avoid  the "ModuleNotFoundError: No module named 'util'" Python Exception.

Also, as stated in the PSC README, I've placed the following lines right under the beginning of def collect_events(...) function:

def collect_event(helper, ew):
    import exec_anaconda
    exec_anaconda.exec_anaconda()
    import pandas
    ...

I keep getting the error: "ERROR Error encountered while loading Python for Scientific Computing, see search.log." But obviusly the search.log file is empty since this is not a SPL search.

Is it possible to use the PSC libraries inside my modular input to accomplish this?

Thank you.