All Apps and Add-ons

Unable to retrieve data from Splunk

pratapa
Explorer

In our environment Nagios and Splunk are integrated. We configured an alert in Nagios monitoring tool which fetches data from Splunk but in Nagios monitoring tool, it is showing as "UNKNOWN - Error in Application name "wms" ".

The alert is configured in such that it is using the script check_splunk_savedsearch_value.sh and it is taking three arguments.

check_splunk_savedsearch_value.sh -a wms -s "WMOS - EW - Number of Allocation records" -w 1

[root@nagios server]# ./check_splunk_savedsearch_value.sh -a wms -s "WMOS - EW - Number of Allocation records" -w 1
UNKNOWN - no output returned from splunk.ce.corp|"wms:WMOS - EW - Number of Allocation records"=ERROR

When we ran the script in debugging mode, the following command is not returning any output.

[root@nagios server ]# /usr/bin/curl -s -k -u username:password https://splunk.ce.corp:8089/servicesNS/monitor/wms/search/jobs/export -d 'search=savedsearch %22WMOS%20%2d%20EW%20%2d%20Number%20of%20Allocation%20records%22' -d output_mode=csv|sed 1d
[root@nagios server]#

Saved search "WMOS - EW - Number of Allocation records" does exist on that search head.

When we click on the saved search, Following is the error it is showing on Web page.

Error in 'script': Getinfo probe failed for external search command 'dbquery'

The search job has failed due to an error. You may be able view the job in the Job Inspector.

Following is the search query.

| dbquery "wmsewprd" "select count(*) from alloc_invn_dtl where stat_code = 11 and (sysdate - mod_date_time) * 24 * 60 > 30"

When we remove | from the above search query it is showing events. Is the | in the above search query making difference ?
If No, what could be the problem.

dbquery "wmsewprd" "select count(*) from alloc_invn_dtl where stat_code = 11 and (sysdate - mod_date_time) * 24 * 60 > 30"

0 Karma
Get Updates on the Splunk Community!

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...