All Apps and Add-ons

Duplicate Alerts

Path Finder

Install of the SolarWinds add-on has gone pretty smoothly other than dealing with the standard SSL cert issues that it seems everyone with SolarWinds is fighting. Querying for alerts is working but no matter what time frame I set for the Checkpoint value or the interval there are duplicate EventIDs being ingested.

Anyone have any ideas on how this could be happening? The version of Splunk is 7.0.0.

Thanks,
Alex

Explorer

Error Log looks like this:
2019-05-22 08:49:17,904 +0000 loglevel=WARNING, pid=23205, tid=Thread-4, file=ext.py, funcname=timestr2str, codelineno=306 | [stanzaname="solarwindsalertstanza"] Unable to convert datestring "2019-05-22T08:41:31.5970000" from format "%Y-%m-%dT%H:%M:%S.%f" to "%Y-%m-%d %H:%M:%S.%f", return the original datestring, cause=Traceback (most recent call last):

Conversion of date_string doesn't work properly as there are too many leading zeros in the date string. the parameter %f only allows 6 digits and not 7.

The fix I found that works for this is to edit the SplunkTASolarWinds/bin/solarwinds_alerts.cc.json line 37.
Change from "%Y-%m-%dT%H:%M:%S.%f" to "%Y-%m-%dT%H:%M:%S.%f0".

0 Karma

Engager

How did u get rid of the SSL cert issue, i know there could be multiple ways, how did you fix it?

Thanks much in advance,
Majid

Splunk Employee
Splunk Employee

Same question!

0 Karma

Splunk Employee
Splunk Employee

Can you take a sapshot from inputs tab? How many inputs do you have there? Are they all enabled?

0 Karma

New Member

Found this message in the _internal logs

index=internal sourcetype="splunktasolarwindssolarwinds_alerts*"

2017-11-13 21:42:21,370 +0000 loglevel=WARNING, pid=19675, tid=Thread-4, file=ext.py, funcname=timestr2str, codelineno=306 | [stanzaname="swalerts"] Unable to convert datestring "2017-11-13T02:59:15.0070000" from format "%Y-%m-%dT%H:%M:%S.%f" to "%Y-%m-%d %H:%M:%S.%f", return the original datestring, cause=Traceback (most recent call last): File "/data/opt/splunk/etc/apps/SplunkTASolarWinds/bin/splunktasolarwinds/cloudconnectlib/core/ext.py", line 289, in timestr2str dt = datetime.strptime(datestring, fromformat) File "/data/opt/splunk/lib/python2.7/strptime.py", line 335, in _strptime datastring[found.end():]) ValueError: unconverted data remains: 0

0 Karma

New Member

We were able to resolve the warning message above by editing the following:

SplunkTASolarWinds/bin/solarwinds_alerts.cc.json

Line 37: "%Y-%m-%dT%H:%M:%S.%f0",
Line 38: "%Y-%m-%d %H:%M:%S.%f0"

Unfortunately the add-on is still pulling in hundreds of thousands of duplicate Events

0 Karma

Path Finder

Ever figure out why you were getting duplicate logs? I had this configured properly once, but during a recent reinstall I forgot what had been configured.

0 Karma

New Member

Per the Python documentation here, the strptime function for the microseconds field (%f) supports 1-6 characters. It looks like our Solarwinds EventTime field is returning 7 places on the microseconds. I tested this and found that removing 1 digit from the microseconds doesn't generate the error.

from datetime import datetime
dt7 = "2017-11-13T04:22:33.1100000"
dt6 = "2017-11-13T04:22:33.110000”
dt6object = datetime.strptime(dt6, "%Y-%m-%dT%H:%M:%S.%f")
dt7
object = datetime.strptime(dt7, "%Y-%m-%dT%H:%M:%S.%f")
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.6/strptime.py", line 328, in _strptime
data
string[found.end():])
ValueError: unconverted data remains: 0

0 Karma

New Member

Hi,

I work on the same system as OP - I'm linking a screenshot of the just the 1 alerts input we have configured and a stats command showing the duplicate EventIDs we're pulling in...

Thanks
Mark

https://imgur.com/a/2Srbq

0 Karma