I am unable to push shcluster bundles post an upgrade to 9.0.2 from 8.2.7. I have also completed the upgrade and migrated the KVstore without error and see the following expected settings: serverVersion : 4.2.17 storageEngine : wiredTiger   The error I receive is: "Error in pre-deploy check, uri=https://<HOST_NAME>/services/shcluster/captain/kvstore-upgrade/status, status=502, error=No error" If I look in splunkd.log I get the following error for each attempt. HttpClientRequest [2071959 TcpChannelThread] - Caught exception while parsing HTTP reply: Unexpected character while looking for value: '<' The error from the actual command makes me think that there was an issue with the kvstore-upgrade that is just not showing. ... View more

I've got a log file that I am monitoring and where I am using a props.conf on the UF to monitor. I'm using the following settings: UF - props.conf: [my_sourcetype] INDEXED_EXTRACTIONS = JSON   Search head cluster  (via deployer in an app bundle) props.conf: [my_sourcetype] KV_MODE = NONE AUTO_KV_JSON = FALSE   If I run btool on one of the search heads for that sourcetype I get: ADD_EXTRA_TIME_FIELDS = True ANNOTATE_PUNCT = True AUTO_KV_JSON = FALSE BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE_DATE = True CHARSET = UTF-8 DATETIME_CONFIG = /etc/datetime.xml DEPTH_LIMIT = 1000 DETERMINE_TIMESTAMP_DATE_WITH_SYSTEM_TIME = false HEADER_MODE = KV_MODE = NONE LB_CHUNK_BREAKER_TRUNCATE = 2000000 LEARN_MODEL = true LEARN_SOURCETYPE = true LINE_BREAKER_LOOKBEHIND = 100 MATCH_LIMIT = 100000 MAX_DAYS_AGO = 2000 MAX_DAYS_HENCE = 2 MAX_DIFF_SECS_AGO = 3600 MAX_DIFF_SECS_HENCE = 604800 MAX_EVENTS = 256 MAX_TIMESTAMP_LOOKAHEAD = 128 MUST_BREAK_AFTER = MUST_NOT_BREAK_AFTER = MUST_NOT_BREAK_BEFORE = SEGMENTATION = indexing SEGMENTATION-all = full SEGMENTATION-inner = inner SEGMENTATION-outer = outer SEGMENTATION-raw = none SEGMENTATION-standard = standard SHOULD_LINEMERGE = True TRANSFORMS = TRUNCATE = 10000 detect_trailing_nulls = false maxDist = 100 priority = sourcetype = termFrequencyWeightedDist = false   I'm not sure what I am missing but I can't get the duplicate field values to cease. ... View more

I have a scheduled search running that uses the Jira JQL search command. Every so often, the search will hang indefinitely. When I look in the job inspector, the only part of the query that ever takes any significant time is the command.jira component. When the job hangs, the command.jira component shows no duration (oldest stalled search was months old and they usually complete in a few minutes at most. My suspicion is that sometimes the jira search is not returning a value, or it's getting missed somehow, and the job is stuck waiting for the results. The problem is that, after a few of these don't complete, the search stops running until I clear out the old failed ones. Has anyone seen similar behavior? Is there a way to configure timeouts for this so that it will exit gracefully instead of hanging? I've been unable to determine anything from the configs or the documentation (seems a bit sparse for this add-on). Thanks for any help. ... View more