All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

By 7pm, I checked the daily quota usage, that it has already reached ~115% usage of the quota, and then over last 30 days, almost every day we use 140% of the Splunk license limit.   However, logs ... See more...
By 7pm, I checked the daily quota usage, that it has already reached ~115% usage of the quota, and then over last 30 days, almost every day we use 140% of the Splunk license limit.   However, logs ingesting in the period of 8pm-00am are still being seen. It does not seem to discard the logs after it exceeds. What is the harm if the indexing ingestion limit reaches?
Hi There,  we have two inputlook kv (File1 and File2)  files and I want to compare 3 columns (AvsA, BvsB, CvsC) between each file and display the result where we clouldn't find any value in all the ... See more...
Hi There,  we have two inputlook kv (File1 and File2)  files and I want to compare 3 columns (AvsA, BvsB, CvsC) between each file and display the result where we clouldn't find any value in all the 3 columns( like non of the comparison written any value I mean blank).  Eg. File1 has Column A which contains roll numbers, Column B contains the Name and Column C contains the Registration number. File2 has the similar columns therefore we need to compare the information (AvsA, BvsB, CvsC) and find out where we have blanks (like there were no results for all the comparison AvsA, BvsB and CvsC) and report them in a table unique once. 
Hi I would like to send search results from Splunk search to my corporate Outlook domain email address, but it is not working. I specified the Outlook SMTP server & port (smtp.office365.com:587) an... See more...
Hi I would like to send search results from Splunk search to my corporate Outlook domain email address, but it is not working. I specified the Outlook SMTP server & port (smtp.office365.com:587) and  user credentials of company domain at the Splunk email settings, but the email was not sent. The command used in the search is this.   index=_internal | sendemail to="myemailid@companydomain" server=smtp.office365.com:587   It doesn't matter if the email is sent from Outlook's SMTP server or from Splunk's local host from anywhere, as long as it can be sent. Information about the Splunk I am using : Splunk Enterprise version 9.0.3 Any ideas are welcomed ! And would be appreciated !
Can anyone tell me if we enable SAML authentication on our on prem windows controller that we are still able to login to the controller using local accounts ? The concern is once SAML is configured ... See more...
Can anyone tell me if we enable SAML authentication on our on prem windows controller that we are still able to login to the controller using local accounts ? The concern is once SAML is configured and you logout and you can no longer login using a local account, how do we revert back to  the "Authentication Provider" : Appdynamics  in which uses local accounts? This tells me that you need to have 2  browser sessions open when configuring SAML:  1st session to be logged in as an Owner role and the 2nd session to enable SAML.  If  the 2nd session fails to login as SAML based user, then in the 1st session we have the ability to set "Authentication Provider"  as Appdynamics   again.  This is to avoid locking out ourselves completely.
Hi..  Spent some one or two hrs, but no luck, hence posting here.. the sample logs: 1.1.1. test log a 1.1.1. test log abc 1.1.2. test log b 1.1.2. test log bcd one thing - no need to worry about ... See more...
Hi..  Spent some one or two hrs, but no luck, hence posting here.. the sample logs: 1.1.1. test log a 1.1.1. test log abc 1.1.2. test log b 1.1.2. test log bcd one thing - no need to worry about timestamps.  one log will have around 10 events. the lines with the starting 1.1.1. should be one event, 1.1.2. should be next event and so on..  or simply, the empty lines should break the events.  please suggest, thanks. 
I am trying to work through an issue and cannot seem to find a answer. I need to create a bash script that uses an auth token to make calls to the Splunk Rest API. The script will be used by a custom... See more...
I am trying to work through an issue and cannot seem to find a answer. I need to create a bash script that uses an auth token to make calls to the Splunk Rest API. The script will be used by a custom app that is located on a searchhead (linux). Token Authentication has been enabled, and I can verify that the Token is enabled and valid.  I can use the Token to make ad-hoc Rest API requests via the terminal local to the searchhead. If I try to use it within the script I am seeing unexpected behavior.  The token seems to be able to make certain API calls in the script but it cannot retrieve the results of the call. An example would be I can use the token in the script to initiate a search. I will receive back the Sid of that search. If I use the same script to try to retrieve the search results using the Sid, I get back nothing. No error message, but just no output at all. If I manually re-run the same API request using the Token and the same Sid outside of the script on the terminal. It will return my search result as expected.  I have turned on debug logging for JsonWebTokenHandler and see no issue. This behavior happens if I use the bash interpreter to execute the script (./script.sh) or (/opt/splunk/etc/app/app_name/bin/script.sh). There is no difference if I execute the script with /opt/splunk/bin/splunk cmd <path_to_script>.  I remember reading that there is something strange about the local interpreter that could be causing this, but I cannot find it again on google. Does anybody know what might be causing this behavior or how I can overcome this issue? Searchhead is running Splunk version 8.1.2 hosted on Ubuntu. 
My Splunk on Prem server's web access randomly shuts off. When attempting to restart the service it just says Splunkd is already running. When attempting to check via Splunk mgmt access... it sho... See more...
My Splunk on Prem server's web access randomly shuts off. When attempting to restart the service it just says Splunkd is already running. When attempting to check via Splunk mgmt access... it shows Splunkwebserver = 0 but the conf shows Splunkwebserver = 1.  
Splunk UF does the load balancing for based on frequency/time. Does load balancing on UF works for file-based inputs as well? Or does that requires external HWF 
I'm using a pretty straightforward query to see how many unique HTTP status codes are thrown from an IIS server during a given time period:   index=foo host=bar sourcetype=iis85 | top sc_status   ... See more...
I'm using a pretty straightforward query to see how many unique HTTP status codes are thrown from an IIS server during a given time period:   index=foo host=bar sourcetype=iis85 | top sc_status   The results are as follows:   sc_status count percent 401 95115 36.975773 302 91840 35.702623 200 70141 27.267179 404 140 0.054425   Adding the percentage values for each of the status codes together equals 100%. I'd like to run the search across a longer timeframe (24 hours) and visualize on a line chart the percentages for each status code using a five minute aggregation. How can I accomplish this?
I'm trying to use a Python script with a custom module for a external lookup on Splunk. When running /opt/splunk/bin/splunk cmd python /opt/splunk/etc/apps/search/bin/gib_detect.py to test the ... See more...
I'm trying to use a Python script with a custom module for a external lookup on Splunk. When running /opt/splunk/bin/splunk cmd python /opt/splunk/etc/apps/search/bin/gib_detect.py to test the script I get the following error:   Traceback (most recent call last): File "/opt/splunk/etc/apps/search/bin/gib_detect.py", line 18, in <module> import gib_detect_train ModuleNotFoundError: No module named 'gib_detect_train'   But when running the same script outside Splunk folders with /opt/splunk/bin/splunk cmd python /home/myuser/gib_detect.py It works as intended. What I am doing wrong?
Hello All, Using the below code, I get the defined quota limits for each role in Splunk environment: -   |rest /services/authorization/roles splunk_server=local   I get below fields related to ... See more...
Hello All, Using the below code, I get the defined quota limits for each role in Splunk environment: -   |rest /services/authorization/roles splunk_server=local   I get below fields related to search quota: - srchJobsQuota rtSrchJobsQuota imported_srchJobsQuota imported_rtSrchJobsQuota Is it possible to know the current utilization from each defined search quota for a given user? If yes, I would need your help to fetch the same. Thank you
Hello everyone, I hope you're doing well. I'm reaching out to this forum for some support regarding an issue I'm facing with Splunk ITSI. Specifically, I'm having trouble gettin... See more...
Hello everyone, I hope you're doing well. I'm reaching out to this forum for some support regarding an issue I'm facing with Splunk ITSI. Specifically, I'm having trouble getting the service_name field to show up in the summary index. I have already tried updating the lookup, but unfortunately, the service_name field is still not being displayed. The service_name field is essential for my analysis, and its absence is hindering my progress. I have taken the following steps to address the issue: Updated the lookup: I made sure to modify the lookup associated with the summary index to include the service_name field. However, despite this update, the field is still not appearing. Despite my efforts, I haven't been able to identify the root cause of the problem. Therefore, I'm turning to this community in the hopes that someone might have encountered a similar issue or can offer guidance on how to resolve it. If anyone has any suggestions, insights, or possible solutions to help me display the service_name field in the summary index, I would greatly appreciate it. Any troubleshooting steps, configuration changes, or alternative approaches you can recommend would be of tremendous help. Thank you in advance for your assistance. Your expertise and support are invaluable to me, and I look forward to any advice or suggestions you can provide. Best regards, AHAR
Hello, Please I need assistance. More than 300 people received a certain email. Some are still with the company while some are not. I need a query that can help me find the total number of people t... See more...
Hello, Please I need assistance. More than 300 people received a certain email. Some are still with the company while some are not. I need a query that can help me find the total number of people that are still with the company that receive this email. Please.  Thank you
Is there a way to specify which metrics should the machine agent send to the controller; for example If I have server visibility enabled and just want to see volume metrics for a server; can I edit a... See more...
Is there a way to specify which metrics should the machine agent send to the controller; for example If I have server visibility enabled and just want to see volume metrics for a server; can I edit any xml file to achieve this? Thanks!
Hello,  I'm reading the Forwarder Management manual and it states " Do not install the universal forwarder over an existing installation of full Splunk Enterprise." What does this mean? My goal is... See more...
Hello,  I'm reading the Forwarder Management manual and it states " Do not install the universal forwarder over an existing installation of full Splunk Enterprise." What does this mean? My goal is to install a universal forwarder on a Linux host, to monitor its /var/log directory.  However, the host has the Splunk search head server installed on it.   Can this be done, without crashing the search head server?  
Hello all. I have a log file that looks like this;   PROCESS UP STATUS RESTARTS AGE PROCESS1 2/2 Running 0 6d19h PROCESS2aaa 2/2 Completed 0 7d6h PROCESS3 0/1 Running 6 6d19h I am trying t... See more...
Hello all. I have a log file that looks like this;   PROCESS UP STATUS RESTARTS AGE PROCESS1 2/2 Running 0 6d19h PROCESS2aaa 2/2 Completed 0 7d6h PROCESS3 0/1 Running 6 6d19h I am trying to evaluate on the RESTART colum. The length of the process name is not consistent and some files are tab delimited and some are space delimited.  I cant get my rex command to work. Any help would be very appreciated. 
Hi, I receive alerts about : DMC Alert - Missing forwarders :   | inputlookup dmc_forwarder_assets | search status="missing" | rename hostname as Instance   it's telling me 3 forward... See more...
Hi, I receive alerts about : DMC Alert - Missing forwarders :   | inputlookup dmc_forwarder_assets | search status="missing" | rename hostname as Instance   it's telling me 3 forwarders are missing, old forwarders since the version is 8.2.1 and we are in 9.0.4.   If I search | inputlookup dmc_forwarder_assets then the result is good, and there are no missing instances anymore ! Where are the 3 missings forwarders actually coming from ? Regarding the Rebuild forwarder assets in the MC does not change anything since the .csv /opt/splunk/etc/apps/splunk_monitoring_console/lookups/dmc_forwarder_assets.csv is showing the right result. My issue is the alerts reporting wrong informations. Any ideas ? Ofc I can provide further informations. Thanks
Hello Splunkers,   I would like to log event for Splunk alert  to an index . If Splunk alert returns only one result then there is no issue. But if there are multiple result it logs only one resu... See more...
Hello Splunkers,   I would like to log event for Splunk alert  to an index . If Splunk alert returns only one result then there is no issue. But if there are multiple result it logs only one result randomly. I have even chosen trigger alert for each result. Any help on this would be great . TIA 
Hi Splunkers, I could have the following scenario to implement: a SplunkCloud platform that must receive data from some data sources, with at least an HF (but they could be 2 or more). Between data... See more...
Hi Splunkers, I could have the following scenario to implement: a SplunkCloud platform that must receive data from some data sources, with at least an HF (but they could be 2 or more). Between data sources, there are Windows Acive Directory servers and, of course, I have to send data to Splunk. I don't know if this data will be routeed to HF or directly to Splunk Cloud, but this is not important now. The focal point is: it is very likely that we will not be able to install UF on those servers, for policy contraints. So, we will have to senda data from AD to Splunk in a agentless manner; what are the possible way to achieve this? And which ones could be the best ones?
Hello Splunkers,  I am facing a problem with my indexers that are not able to index anymore. Neither the data forwarder to those indexers, neither the internal Splunk logs... I even tried to index... See more...
Hello Splunkers,  I am facing a problem with my indexers that are not able to index anymore. Neither the data forwarder to those indexers, neither the internal Splunk logs... I even tried to index data (simple txt file) directly from the indexer GUI, I do not get any error but my selected indexe will not be filled/updated. Any clue what I can do to troubleshoot ? There is nothing in splunkd.log file, what other logs should I check? Regards, GaetanVP