Splunk Enterprise Security

How to add Python librarys to Splunk with PyUnicode UCS4


I created an custom command with iocextract Python libray inside a new Splunk app. https://github.com/InQuest/python-iocextract
sudo pip -t . iocextract
With this command I installed the libary iocextract and all the required libraries regex + ipadress successfully.
My script imports iocextractlibrary.iocextact` imports the regex library but this library has an encoding problem.

The script:

#!/usr/bin/env python

import sys
import os
import iocextract

from splunklib.searchcommands import \
    dispatch, StreamingCommand, Configuration, Option, validators

class extractiocCommand(StreamingCommand):
    """ %(synopsis)

And when I search with index="_internal" |head 10 |table source |eval text="text" | extractioc fieldname=text kind=urls

I get this error message:

Error in 'script': Getinfo probe failed for external search command 'extractioc'.
from .regex import *
import _regex
import _regex_core
import iocextract
import regex as re
File "/opt/splunk/etc/apps/IocExtractor3/bin/extractioc.py", line 5, in <module>
File "/opt/splunk/etc/apps/IocExtractor3/bin/iocextract.py", line 10, in <module>
File "/opt/splunk/etc/apps/IocExtractor3/bin/regex/__init__.py", line 1, in <module>
File "/opt/splunk/etc/apps/IocExtractor3/bin/regex/_regex_core.py", line 21, in <module>
File "/opt/splunk/etc/apps/IocExtractor3/bin/regex/regex.py", line 391, in <module>
ImportError: /opt/splunk/etc/apps/IocExtractor3/bin/regex/_regex.so: undefined symbol: PyUnicodeUCS4_FromUnicode
Traceback (most recent call last):
The search job has failed due to an error. You may be able view the job in the Job Inspector. 

I tested my python encoding inside the folder /opt/splunk/etc/apps/IocExtractor3/bin# python

import sys
if sys.maxunicode > 65535:
 print 'UCS4 build'
 print 'UCS2 build'

and get UCS4 build

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...