SolarWinds app modular error.

[NHLaurent]

Hi All,

I've installed the latest SolarWinds Add-on for Splunk (https://splunkbase.splunk.com/app/3584/#/details)

After installing and restart I'm getting the following error;
Unable to initialize modular input "solwarwinds_query" defined in the app "Splunk_TA_SolarWinds": Introspecting scheme=solwarwinds_query: script running failed (exited with code 1)..

When opening the app it just sits there spinning its logo.

Any ideas

Splunk Ent V8 on WIndows.

[loconnor]

I got this to work on Splunk 8.1.1 by adding:

python.version = python2

to all sections in inputs.conf and restmap.conf. Could probably add it to a [default] section in both of those, but it certainly works by adding it to the individual stanzas (in local.)

[rupert]

I tried this method and it works! 

I added the following to all the input stanzas in local.

I also added it into restmap.conf (in local) but encountered error saying that the parameter is not valid for the stanzas.

So I guess just by adding to the inputs.conf it is working. 

python.version = python2

Both the UI of the add-on and the log collection are working fine.

I am using SE 8.1.0 with Solarwinds Add-on 1.1.0

and I configured the add-on for inventory and alert inputs.

Thank you very much to @loconnor 

[mfortn_22]

@rupert @loconnor Could you expand on which specific inputs.conf file(s) you added the 'python.version = python2' statement too. Would this be within the splunk_TA_solarwinds folder or another like the system folder? One of out admins upgraded last night and we too are running into this.

[ehaddad_splunk]

should be all stanzas under inputs.conf and restmap.conf. Should have an updated version with a fix that would not require this workaround. Send me an email and I can you an updated copy of the TA. Otherwise, i might make it available on splunkbase soon.

[loconnor]

As I showed in my reply, I decided to create a [default] section that will be applied to all inputs and restmap stanzas, provided they don't have their own override.

[loconnor]

lance@mono Splunk_TA_SolarWinds % head -4 local/restmap.conf local/inputs.conf
==> local/restmap.conf <==
[default]
python.version = python2

==> local/inputs.conf <==
[default]
python.version = python2
index = solarwinds

lance@mono Splunk_TA_SolarWinds %

[mfortn_22]

Thank you! I was heading down the correct path.  @ehaddad_splunk I sent you a pm for the update

[rupert]

Hi @mfortn_22 , I am sorry I didn't get to read your message last week.

Thanks @loconnor  again for the workaround and details in the conf.

Yes, I think the parameter needs to be added to both input.conf and restmap.conf. The error I saw was only for [admin:] stanza in restmap.conf. The settings in other stanzas (or a [default] setting) is still required for it to work.

[cchacon]

It may just be a simple TA update to python3 then.

Python2 is deprecated from my understanding. Good to know!

[cchacon]

Would be nice for the Splunk Devs to confirm if this TA will work on Enterprise 8.0+ or the latest ITSI version. My understanding is that the add-on will log inputs for SolarWinds query, alerts, or node inventory (esp useful with the major SolarWinds breach announced by Homeland Security as of Dec 14 2020).

Is this only for cloud or is it supported, as stated in release notes, for all tiers of Splunk? Looking for method of monitoring SolarWinds once they remove the Trojan DLL from their repo.

[hansuleberg]

Same issue here as well.

Unable to initialize modular input "solwarwinds_query" defined in the app "Splunk_TA_SolarWinds": Introspecting scheme=solwarwinds_query: script running failed (exited with code 1)..

Splunk Enterprise 8.1 on Linux.

[PG]

I notified Splunk via a ticket and they told me I needed to contact Solarwinds.  I then opened a ticket with Solarwinds and they stated I need to go back to Splunk.  So I opened another ticket and they told me 

"something that I am noticing is that the Add-On is only compatible to Splunk Cloud environment and this ticket has been open as an On-Prem instance. Splunk Enterprise and Splunk Cloud differ significantly in software composition, if you are installing in Splunk Enterprise(On-Prem) then the installation will definitely not work and that's the reason of this issue."

We are on Splunk 8.1 on Linux as well.  We use AWS Linux servers.  We had the add on working before with no issues.  Not sure if this helps any.

[cchacon]

Have you been able to configure Splunk_TA_SolarWinds for Linux Splunk on-prem? We use the same 

[hansuleberg]

Thank you.

Yes, what is strange it is listed as supported on Splunk platform 8.1

https://splunkbase.splunk.com/app/3584/#/details

COMPATIBILITY

Products: Splunk Enterprise, Splunk Cloud

Splunk Versions: 8.1, 8.0 

Platform: Platform Independent

We were on where Solarwinds AddOn worked fine:

- AWS Linux
- Splunk Enterprise 8.0.5
- Solarwinds addon version 1.1.0

Upgraded to:

- AWS Linux
- Splunk Enterprise 8.1
- Solarwinds addon version 1.1.0

So something must be wrong in the python code.
This is the line 388 it complain about in our splunkd.log file:

def validate(self, value, data):
import datetime
try:
datetime.datetime.strptime(value, self._format)
except ValueError, exc:
error = 'Wrong datetime with format "%s": %s' % (self._format, str(exc))
self.put_msg(error)
return False
return True

[rmorlen]

You are seeing that the app is supposed to be compatible.  It is not a Splunk Supported addon.

[hansuleberg]

Hi.

But it has passed the splunk app inspect, and is listed as supported on Splunk Enterprise 8.1 and 8.x

I have a support ticket engineer looking into it now inside splunk.

[asearson]

We are also encountering this issue. Addon works fine with Splunk Enterprise 8.0.x, but fails when upgrading (or even fresh installs) on 8.1.x.

We also opened a Support ticket, but was immediately Rejected for the Unsupported App:

Having said that, there does appear to be a bit of interest from the Community and Splunk answers so hopefully the developers will see that. I note the latest version 1.1.0 was released in March 2020, well before Splunk 8.1, so as you have suggested it's not finding the right python libraries etc.

Looks like the author @ehaddad_splunk  (?) is our only hope for an update.

[jaxjohnny2000]

@ehaddad_splunk   Is there a limit to the structure or size of the SWQL?  

I can get Alerts, inventory, and simple queries no issue.  But when I run a complex search, I get status=400

Should this be a new topic? 

2021-03-18 11:43:11,872 +0000 log_level=ERROR, pid=30166, tid=Thread-4, file=engine.py, func_name=_send_request, code_line_no=325 | [stanza_name="test_001"] The response status=400 for request which url=https://10.1.2.21:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT IPAddress1, IPAddress2, CASE WHEN IPAddress1 IS NULL THEN NULL ELSE H1.Hostname END AS Hostname1, CASE WHEN IPAddress2 IS NULL THEN NULL ELSE H2.Hostname END AS Hostname2, TotalBytesIngress, TotalPacketsIngress, TotalBytesEgress, TotalPacketsEgress, TotalBytesIngress + TotalBytesEgress AS TotalBytes, TotalPacketsIngress + TotalPacketsEgress AS TotalPackets FROM (SELECT TOP 10 SourceIP AS IPAddress1, DestinationIP AS IPAddress2, MAX(SourceHostnameID) AS HostnameID1, MAX(DestinationHostnameID) AS HostnameID2,SUM(IngressBytes) AS TotalBytesIngress, SUM(IngressPackets) AS TotalPacketsIngress, SUM(EgressBytes) AS TotalBytesEgress, SUM(EgressPackets) AS TotalPacketsEgress, SUM(IngressBytes) + SUM(EgressBytes) AS TotalBytes, SUM(IngressPackets) + SUM(EgressPackets) AS TotalPackets FROM Orion.Netflow.FlowsByConversation Flows WHERE (Timestamp >= (GetUTCDate() - 0.04167)) GROUP BY (SourceIP, DestinationIP) ORDER BY TotalBytes DESC) OuterFlows LEFT JOIN Orion.Netflow.Hostnames AS H1 ON H1.ID = OuterFlows.HostnameID1 LEFT JOIN Orion.Netflow.Hostnames AS H2 ON H2.ID = OuterFlows.HostnameID2 ORDER BY TotalBytes DESC, IPAddress1 ASC, IPAddress2 ASC and method=GET.


However, this one works fine:

2021-03-18 11:55:37,398 +0000 log_level=INFO, pid=12462, tid=Thread-4, file=http.py, func_name=request, code_line_no=169 | [stanza_name="test_002"] Invoking request to [https://10.1.2.1:17778/SolarWinds/InformationService/v3/Json/Query?query=SELECT%20Caption%20AS%20NodeName,%20IPAddress%20FROM%20Orion.Nodes] finished

[ehaddad_splunk]

I havent tested it against 8.1. I will need to take a look and update accordingly. 

The addon is only needed on the HWF - any chance you can keep the addon on an 8.0x forwarder?  

[rupert]

Hi,

We are encountering same issue here with the Solarwinds add-on, after upgrading

from 

SE 7.3.1 + Solarwinds Add-on 1.0.0

to

SE 8.1.0 + Solarwinds Add-on 1.1.0

Both versions of add-on are not working with the new version of SE. The configuration UI cannot be loaded and the modular input is also not pulling anything from Solarwinds.

May I check if there is any update to this issue please? Will there be an update of the add-on which is compatible with SE 8.1.x please?

Thank you