Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

secret_lang — Tue, 09 Mar 2021 04:33:30 GMT

Issue importing pandas module into Splunk Add-on Builder

import re import sys import os ta_name = 'TA-urlparse' ta_lib_name = '/usr/local/lib/python3.7/site-packages' pattern = re.compile(r"[\\/]etc[\\/]apps[\\/][^\\/]+[\\/]bin[\\/]?$") new_paths = [path for path in sys.path if not pattern.search(path) or ta_name in path] new_paths.insert(0, os.path.sep.join([ta_lib_name])) sys.path = new_paths import numpy as np import pandas import requests import ipaddress import time from datetime import datetime

The Error:

Traceback (most recent call last): File "/apps/splunk/etc/apps/TA-urlparse/bin/urltest_1615256627_866.py", line 14, in <module> import input_module_urltest_1615256627_866 as input_module File "/apps/splunk/etc/apps/TA-urlparse/bin/input_module_urltest_1615256627_866.py", line 16, in <module> import pandas File "/usr/local/lib/python3.7/site-packages/pandas/__init__.py", line 52, in <module> from pandas.core.api import ( File "/usr/local/lib/python3.7/site-packages/pandas/core/api.py", line 29, in <module> from pandas.core.groupby import Grouper, NamedAgg File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/__init__.py", line 1, in <module> from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 57, in <module> from pandas.core.aggregation import ( File "/usr/local/lib/python3.7/site-packages/pandas/core/aggregation.py", line 27, in <module> from pandas.core.series import FrameOrSeriesUnion, Series File "/usr/local/lib/python3.7/site-packages/pandas/core/series.py", line 68, in <module> from pandas.core import algorithms, base, generic, nanops, ops File "/usr/local/lib/python3.7/site-packages/pandas/core/generic.py", line 102, in <module> from pandas.io.formats import format as fmt File "/usr/local/lib/python3.7/site-packages/pandas/io/formats/format.py", line 71, in <module> from pandas.io.common import stringify_path File "/usr/local/lib/python3.7/site-packages/pandas/io/common.py", line 7, in <module> import mmap ModuleNotFoundError: No module named 'mmap'

"Extensive" Troubleshooting:

/usr/bin/python3 -m pip install mmap ERROR: Could not find a version that satisfies the requirement mmap ERROR: No matching distribution found for mmap

Per: https://community.splunk.com/t5/All-Apps-and-Add-ons/how-can-I-enable-MMAP-caching-in-Google-GeoIP-of-MaxMind/m-p/33754

copied mmap.so and am now getting the following:

Traceback (most recent call last): File "/apps/splunk/etc/apps/TA-urlparse/bin/urltest_1615264087_827.py", line 14, in <module> import input_module_urltest_1615264087_827 as input_module File "/apps/splunk/etc/apps/TA-urlparse/bin/input_module_urltest_1615264087_827.py", line 16, in <module> import pandas File "/usr/local/lib/python3.7/site-packages/pandas/__init__.py", line 52, in <module> from pandas.core.api import ( File "/usr/local/lib/python3.7/site-packages/pandas/core/api.py", line 29, in <module> from pandas.core.groupby import Grouper, NamedAgg File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/__init__.py", line 1, in <module> from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy File "/usr/local/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 57, in <module> from pandas.core.aggregation import ( File "/usr/local/lib/python3.7/site-packages/pandas/core/aggregation.py", line 27, in <module> from pandas.core.series import FrameOrSeriesUnion, Series File "/usr/local/lib/python3.7/site-packages/pandas/core/series.py", line 68, in <module> from pandas.core import algorithms, base, generic, nanops, ops File "/usr/local/lib/python3.7/site-packages/pandas/core/generic.py", line 102, in <module> from pandas.io.formats import format as fmt File "/usr/local/lib/python3.7/site-packages/pandas/io/formats/format.py", line 71, in <module> from pandas.io.common import stringify_path File "/usr/local/lib/python3.7/site-packages/pandas/io/common.py", line 7, in <module> import mmap ImportError: /apps/splunk/lib/python3.7/mmap.so: undefined symbol: PL_mmap_page_size

Any Ideas?

Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

andreasz — Sat, 07 Aug 2021 23:05:09 GMT

This is what worked for me: download the Python for Scientific Computing app https://splunkbase.splunk.com/app/2882 and copy the following two files

/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python3.7/lib-dynload/mmap.cpython-37m-x86_64-linux-gnu.so

/opt/splunk/etc/apps/Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python3.7/lib-dynload/cmath.cpython-37m-x86_64-linux-gnu.so

to /opt/splunk/lib/python3.7

Tested it with custom search commands.

Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

mzvolsky — Wed, 16 Mar 2022 18:51:02 GMT

In Python for Scientific Computing I see Py3.8 which differs from the version in splunk `bin/splunk cmd python --version`, which was 3.7.10. So I have Py3.7 installed (using `pyenv install 3.7.10` which enables more python versions on the machine) and from ~/.pyenv/versions/3.7.10 I have copied the both missing files into `/opt/splunk/lib/python3.7/lib-dynload`.

The `pip install splunk-sdk` then run well for me.

Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

mzvolsky — Wed, 16 Mar 2022 18:56:20 GMT

to my previous answer (get missing mmap+cmath): .... the same can be sure achieved by selecting older version of `Py for Scientific computing` on the download page; but I have not an idea which one includes Py3.7.

topic Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap' in Splunk Enterprise

Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'

Re: Add-On Builder [pandas]: ModuleNotFoundError: No module named 'mmap'