All Apps and Add-ons

Splunk ISE TA fails when distributed via Cluster Master

bjoernhansen
Path Finder

Hey,

I use this TA on an Indexer cluster, and it fails to actually process any event. After some troubleshooting, the core of the problem lies here:

05-05-2017 16:49:01.514 +0200 ERROR AggregatorMiningProcessor - Uncaught exception in Aggregator, skipping an event: Can't open DateParser XML configuration file "/opt/splunk/etc/apps/Splunk_TA_cisco-ise/default/datetime_udp.xml": No such file or directory - data_source="/opt/logfiles/10.1.2.3/2017-05-05.log", data_host="10.1.2.3", data_sourcetype="cisco:ise:syslog"

When the app is distributed via the cluster master, it is stored in /etc/slave-apps on the indexers, and the reference in the default/props.conf does not work:

[cisco:ise]
DATETIME_CONFIG = /etc/apps/Splunk_TA_cisco-ise/default/datetime_udp.xml

[cisco:ise:syslog]
DATETIME_CONFIG = /etc/apps/Splunk_TA_cisco-ise/default/datetime_udp.xml

I fixed this by creating a local/props.conf like this:

[cisco:ise]
DATETIME_CONFIG = /etc/slave-apps/Splunk_TA_cisco-ise/default/datetime_udp.xml

[cisco:ise:syslog]
DATETIME_CONFIG = /etc/slave-apps/Splunk_TA_cisco-ise/default/datetime_udp.xml

This obviously only fixes it for cluster environments.

From my understanding, DATETIME_CONFIG allows for no variable that includes the app path, so I see no obvious solution to fix this.
If it can't be fixed, at least an eye-catching hint about this in the install guide could be helpful.

ChrisBell04
Communicator

I too have encountered this with the CM. Finally decided to remove it entirely.

custom prefix stuff for our syslogs, modify to meet your env.

[cisco:ise:syslog]
DATETIME_CONFIG = 
MAX_TIMESTAMP_LOOKAHEAD = 40
NO_BINARY_CHECK = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N %:z
TIME_PREFIX = \S+\s\S+\s\S+\s\d+\s\d+\s\d+\s
0 Karma

highsplunker
Contributor

OMG. Seems i have similar problem. But i'm dealing with heavy forwarder problem now. Nevertheless the error is the same. Not fixed it yet.

0 Karma

jcspigler2010
Path Finder

Just dealt with this today at a customer site. Luckily I tailed the splunkd.logs on one of the indexers while their syslog w/ UF was firing data at this particular indexer. Saw the error and spiraled down to your same conclusion. I submitted feed back on this TA to have the doc site updated with instructions to modify the DATETIME_CONFIG variable in clustered index environments...

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...