All Apps and Add-ons

Add-on for JIRA: Why does the add-on display error "Can not import package:libs_setup_util" when trying to execute using an interval and indexing?

jb_spelunker
Explorer

Add-on for JIRA is installed and executes fine on-demand with config.ini when using things like:

| jira jqlquery "Project='bob'"

However, trying to use the interval based query and index method within inputs.conf it throws this error every execution:

2017-05-05 15:31:20,386 INFO pid=36306 tid=MainThread file=base_modinput.py:__init__:51 | Can not import package:libs_setup_util
2017-05-05 15:31:20,387 ERROR pid=36306 tid=MainThread file=base_modinput.py:log_error:70 | Get error when collecting events.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/dpz_jira_plugin_search/bin/libs/modinput_wrapper/base_modinput.py", line 174, in stream_events

The strange thing is that there is no module called lib_setup_util in the Splunk Python distribution, or the O/S Python distribution, including python-libs

Active internet searches regarding modular input yield nothing aside from a similar Splunk Add-on for Bamboo.

What module, path, or combination of both is required for this import to function?

0 Karma

jb_spelunker
Explorer

Resolved.

1) If you change the name of the Application to meet your organization's naming standards, make sure you change the jira.py line 84 to this:

    return "your_name_here"

2) You don't need the modinput. this whole piece can be commented out of lib/modinput_wrapper/base_modinput.py:

    try:
        self.setup_util_module = importlib.import_module(self.namespace + "_setup_util")
    except ImportError as ie:
        self.logger.info("Can not import package:" +
                         self.namespace + "_setup_util")

3) The Python script appends "and updated > -5m" to the JQL, (5m is based on what inputs.conf has for interval) so make sure you wrap your JQL in ( ) so that it's treated as a complete query and the "and updated" logic gets applied after the JQL.

4) If you use the "fields" limiter in the inputs.conf, you must have "updated" in the list or the jira.py fails. Also, make sure you separate your fields with a , followed by a " " - because the split in jira.py splits based on ", "

Why it requires a space defies the logic behind csv format, but it must be there.

Field limiting and a smart JQL are a must, because otherwise a massive amount of data is returned and your Splunk queries will have to be very selective.

Many things, but this does now work in my environment.

vpham_splunk
Splunk Employee
Splunk Employee

Hey jb_spelunker,

There are 2 issues here:
1. INFO pid=36306 tid=MainThread file=base_modinput.py:init:51 | Can not import package:libs_setup_util
This Add-on use Add-on builder but it didn't define the setup_utils libary. However, this is just INFO log and it shouldn't affect anything nor the second issue.
2. ERROR pid=36306 tid=MainThread file=base_modinput.py:log_error:70 | Get error when collecting events.
I couldn't find any clue from this partial log. Could you post more detail?

As you described, live query | jira works so the connection between your Splunk instance and JIRA server is good. How does your JQL which is defined in inputs.conf look like? Does it a massive query which returns too many records? I know some JIRA administrators define some policies against REST API query. If it is the case, try using queries which returns small amount of data.

0 Karma

jb_spelunker
Explorer

Hello Vu,

It would be good for my own peace of mind to be able to fix the Python code - what is the setup_utils library and where is it maintained (do you know?)

The details per your question ...

jql returns ~500 records fairly quickly (23 seconds), it's a very basic query as we're currently only testing.

from the local folder, inputs.conf looks like this:

[jira://splunk]
sourcetype = jira
interval = 60
server = hostname
protocol = https
port = 443
jql = issuetype in (epic)
username = A
password = B
disabled = 0
index = jira-plugin

jira.conf in the same folder looks like this, roughly the defaults with a default project added and a max record return value defined:

[jira]
default_project = CORE
tempMax = 1000
keys = link,project,key,summary,type,priority,status,resolution,assignee,reporter,created,updated,resolved,fixVersion,components,labels/label
time_keys = timeestimate, originalestimate, timespent

0 Karma

vpham_splunk
Splunk Employee
Splunk Employee

bin/libs/modinput_wrapper/base_modinput.py:46

        try:
            self.setup_util_module = importlib.import_module(self.namespace +
                                                             "_setup_util")
        except ImportError as ie:
            self.logger.info("Can not import package:" +
                             self.namespace + "_setup_util")

I believe it's safe to remove that code since the app doesn't define the setup_util.

0 Karma

vpham_splunk
Splunk Employee
Splunk Employee

Could you paste detailed exception log? i.e what HTTP error code does it return? etc.

0 Karma

jb_spelunker
Explorer

there is no exception aside from the setup_utils - this is the only thing mentioned in the libs_jira.log for every execution:

2017-05-19 09:42:11,593 INFO pid=12703 tid=MainThread file=base_modinput.py:init:50 | Can not import package:libs_setup_util
2017-05-19 09:42:11,594 ERROR pid=12703 tid=MainThread file=base_modinput.py:log_error:69 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/dpz_jira_plugin_search/bin/libs/modinput_wrapper/base_modinput.py", line 173, in stream_events
self.collect_events(inputs, ew)
File "/opt/splunk/etc/apps/dpz_jira_plugin_search/bin/jira.py", line 307, in collect_events
self.extract(inputs)
File "/opt/splunk/etc/apps/dpz_jira_plugin_search/bin/jira.py", line 254, in extract
self.fields = self.input_items['fields']
KeyError: 'fields'

After removing the setup_util lines from the base_modinput.py, the errors stop.

However, there is still no data in the specified indexer after several 60 second intervals have passed.

0 Karma

vpham_splunk
Splunk Employee
Splunk Employee

According to log file, it looks like you haven't specified what fields you want to get in inputs.conf, please do this if you want to get all fields return by payload:
[jira://splunk]
...
fields = *
...

0 Karma

vpham_splunk
Splunk Employee
Splunk Employee

Hi jb_spelunker, does this help?

0 Karma

akarivaratharaj
Communicator

Hi vpham,

I am also facing the same issue. If I run any simple jql query also I am getting the results as "no results found". When I check the libs_jira.log, I am getting the below error:

2017-09-26 15:53:24,792 INFO pid=10288 tid=MainThread file=base_modinput.py:init:50 | Can not import package:libs_setup_util
2017-09-26 15:53:24,795 ERROR pid=10288 tid=MainThread file=base_modinput.py:log_error:69 | Get error when collecting events.
Traceback (most recent call last):
File "C:\Users\akarivarathara\Documents\SRE\Splunk\etc\apps\jira\bin\libs\modinput_wrapper\base_modinput.py", line 173, in stream_events
self.collect_events(inputs, ew)
File "C:\Users\akarivarathara\Documents\SRE\Splunk\etc\apps\jira\bin\jira.py", line 307, in collect_events
self.extract(inputs)
File "C:\Users\akarivarathara\Documents\SRE\Splunk\etc\apps\jira\bin\jira.py", line 249, in extract
self.protocol = self.input_items['protocol']
KeyError: 'protocol'

But I gave correct protocol only as "https". As this is showing the error, I changed it to "http". Even that setting also throwing the same error.
Can you please help me in this case.

0 Karma
Get Updates on the Splunk Community!

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...