Hi, I am using the Splunk Add-on for Box. I have successfully set oauth2 access token and refresh token. The problem I am facing is that the Oauth2 refresh token does not always work and very frequently, I need to authorize the app again manually to make it work. Is this a normal issue with this Splunk add-on? Any workaround for this? ... View more

Hi All , I have developed a mini POC to look out for Splunk End to End Monitoring . The POC will be triggered if there is a missing log source being reported in the splunk alert. Below are my checks and i would like to know that whether i have missed any checks ? Main Query : Splunk Query for missing log sources. This will trigger the below steps: 1. Splunk Connection to Search Head 1.a If splunk connection fails then check for network connection to Search head instance by a 'ping', followed by a health check on ports and services. 2. If connection is successfull, Splunk Query to check whether all indexers are reporting for last say 60 mins. 2.a if some of indexers are not reporting then, check for network connection to indexers with a ping followed by a health check on ports and services. 3. If connection is successfull , then Splunk query to check for Blocked Queues at Indexer level 4. Splunk Query to check for Missing forwarder. 5. If missing forwarder results, then check for forwarder availability with a ping, followed by a check on splunk socket connection and health check on ports and services. 6. Splunk Query to check for data throttling at forwarder level. These are the checks that i have implemented which might cause a missing log source. Checks are only within Splunk Infra. Please let me know if i have missed any checks ... View more

Hi all, i have developed a query to check my host lastphonehoemtime information in DS as below: |rest /services/deployment/server/clients splunk_server=local | eval current = now() | convert ctime(lastPhoneHomeTime) as last ctime(current) as now | fields hostname ip last last now The above query is working fine. Now i want to call this via SDK, so i am using below code for that ds_url = 'https://xyz:8089' username='abc' password='pqr' request = urllib2.Request(ds_url + '/servicesNS/%s/search/auth/login' % (username), data = urllib.urlencode({'username': username, 'password': password})) server_content = urllib2.urlopen(request) session_key = minidom.parseString(server_content.read()).getElementsByTagName('sessionKey')[0].childNodes[0].nodeValue print "Session Key: %s" % session_key search_query=' |rest /services/deployment/server/clients | eval current = now() | convert ctime(lastPhoneHomeTime) as last ctime(current) as now | fields hostname ip last last now' request = urllib2.Request(ds_url + '/services/deployment/server/clients', data = urllib.urlencode({'search': search_query,'output_mode': 'csv'}), headers = { 'Authorization': ('Splunk %s' %session_key)}) search_results = urllib2.urlopen(request) print search_results.read() I have two questions: How should i write the Splunk query to insert in the urllib2 object. Since in the main query 'rest' should be the first parameter and when calling via SDK first parameter should be 'search' ? How to pass value of /services/deployment/server/clients in urllib2.request method as the format should be /services/<user>/<app> ? Any help would be highly appreciated !! ... View more

Hi all, I have written below metadata search to find the hosts which have reported yesterday, but not reporting in the last 1 hour. | metadata type=hosts |eval current=now() |eval lastHour=relative_time(current,"-1h") | eval yesterday=relative_time(now(), "-1d") | where ( recentTime>yesterday AND recentTime<lastHour) | convert ctime(current) as Current_Time, ctime(lastHour) as Last_Hour, ctime(recentTime) as Recent | table Current_Time,Last_Hour,Recent, host This search is returning a list of hosts, but after I dig into these hosts, they seems to be reporting to their respective indexes which means the metadata search is giving me wrong results. I start breaking up the search and excluded the recentTime&lt;lastHour from the where clause. Below is the resulting search: | metadata type=hosts | eval current=now() |eval lastHour=relative_time(current,"-1h") | eval yesterday=relative_time(now(), "-1d") | where recentTime>yesterday | convert ctime(current) as Current_Time, ctime(lastHour) as Last_Hour, ctime(recentTime) as Recent | table Current_Time,Last_Hour,Recent, host This gives me a list of only 2 hosts, whereas in my environment, hundreds of hosts are reporting. Not really sure what is happening. Please help !! ... View more

Hi All , I am trying to find the hosts which have not reported for the last 1 hour, so i am using metadata command. | metadata type=hosts | eval current=now()| eval lastHour=relative_time(now(),"-1h") | where lastHour > recentTime | convert ctime(current) as Current_Time, ctime(lastHour) as Last_Hour, ctime(recentTime) as Recent | table Current_Time,Last_Hour,Recent, host I am using recentTime because it a loge stream. Problem that i am facing is this metadata command shows me all hosts with recentTime before 1 hour. Attached is a screenshot for the same. Mu hosts have reported after that also. Even if i change the limit to 1 day then the search is showing me results for al hosts with recentTime before 1 day. Any help !! alt text ... View more

Hi All , My problem statement is to find the blocked queues over 60 minutes consistently which means that there should be no succesfull connection of queues between blocked queues events. So the approach i am following is to compare the number of blocked queues events with the number of events in indexer for last 60 minutes.if they are equal i will be sure that only blocked queues events exist. First i write the following query to count the events per host for blocked queues search index=_internal earliest=-60m@m source=*metrics.log group=queue "blocked" | stats count AS Number by host Then i need to pass the above calculated hosts value in the mainn search so that only for these host the main search runs index=_internal earliest=-60m@m source=*metrics.log group=queue [ search index=_internal earliest=-60m@m source=*metrics.log group=queue "blocked" | stats count AS Number by host | fields + host ] Above part is giving me correct host values No comes the comparing part, so i build this query index=_internal earliest=-60m@m source=*metrics.log group=queue [ search index=_internal earliest=-60m@m source=*metrics.log group=queue "blocked" | stats count AS Number by host | fields + host ] | stats count as Point by host | append [search index=_internal earliest=-60m@m source=*metrics.log group=queue "blocked" | stats count AS Number by host | fields + Number host ] | table host Number Point which gives me output host Number Point a 123 b 147 a 1 b 2 Output is expected as i am running the stats earlier also i the main query but i cannot figure out, how to compare Number and Point count's. Also please help with the usage of "If" command since if the result matches i need to print the queue name along with host. Please help !! ... View more