I need to upgrade a Search Head Cluster from 7.3.4 to 8.1.9 and I have run the first two commands: splunk upgrade-init shcluster-members splunk edit shcluster-config -manual_detention on We are monitoring the active searches using the following command: splunk list shcluster-member-info | grep "active" And we see: active_historical_search_count:1 active_realtime_search_count:0 And it seemed to never reduce down to 0 for the active_historical_search_count, but after 90 minutes, it seems to have come down to 0. We checked the currently-running searches and found some new searches running on the detention server after 1 hour. We have the following set in the server.conf: decommission_force_finish_idle_time = 0 decommission_node_force_timeout = 300 decommission_search_jobs_wait_secs = 180 ...so why is it taking 90 minutes to stop running savedsearches? We did find some savedsearches that were running for long times and we fixed them, but should not all new searches be moved to another server once it is in manual detention? What can I do to fix this, so that my SHC can be upgraded?

Hey Team, I have some 150+ ip addresses in CIDR format (IE 96.24.0.0/16, etc) , i am getting my search result with one values coming as  dst_ip 96.24.123.123.  i need to filter out this event. so basically if it would be one,, i can simply do in my SPL dst_ip!= (96.24.0.0/16) or NOT dst_ip IN ((96.24.0.0/16),  but i have around 150+ cidr that i need to filter out. i tried to add them into lookup file and it seems cidr in lookfile is not working. can someo

Hello, I have 2 CSVs in my splunk: Alert.csv having below columns and data: Alert_Header   Alert_type   Date JNA/athena_VICTORIA_Load [FAILED]   Autosys   01/03/2022 JNA/athena_VICTORIA_Staging [MAXRUN]   Autosys   01/03/2022 JNA/athena_MAIN_Load [FAILED]   Autosys   01/03/2022 JNA/athena_OLTP_Staging [MAXRUN]   Autosys   01/03/2022 NYP02000 has high_cpu   DATABASE   01/03/2022   Mapping.csv having below columns and data: Alert_Type   Inclusion   Exclusion   Header Autosys   athena   VICTORIA   ATHENA-Jobs Autosys   VICTORIA   NONE   VICTORIA-Jobs Database   high_cpu   NONE   CPU alerts   Output required: Alert_Header   Alert_type   Date   Header JNA/athena_VICTORIA_Load [FAILED]   Autosys   01/03/2022   VICTORIA-Jobs JNA/athena_VICTORIA_Staging [MAXRUN]   Autosys   01/03/2022   VICTORIA-Jobs JNA/athena_MAIN_Load [FAILED]   Autosys   01/03/2022   ATHENA-Jobs JNA/athena_OLTP_Staging [MAXRUN]   Autosys   01/03/2022   ATHENA-Jobs NYP02000 has high_cpu   DATABASE   01/03/2022   CPU alerts   Logic: Mapping file is for looking up pattern in the Alert.csv. So if any Alert_Header which has "athena"(mentioned in Inclusion) but doesn't have "Victoria" (mentioned in Exclusion) keyword in it will be termed as "ATHENA-Jobs" . Similarly if any Alert_Header which has "Victoria"(mentioned in Inclusion) only will be termed as "VICTORIA-Jobs". None in Exclusion column would mean there is no exclusion pattern to be searched in Alert_Header.   Can you please help with this query

As shown below I have only two events present on my index But when i execute the below search query index = **** |rex field=_raw "(?msi)(?<json_field>\{.+\}$)" | spath input=json_field |rename SCMSplunkLog.SCMFailureLog.appName as APPNAME,SCMSplunkLog.SCMFailureLog.eventType as EVENTTYPE,SCMSplunkLog.SCMFailureLog.payload.level as LEVEL,SCMSplunkLog.SCMFailureLog.payload.errorDescription as ERRORDESCRIPTION,SCMSplunkLog.SCMFailureLog.payload.startTime as STARTDATE,SCMSplunkLog.SCMFailureLog.payload.endTime as ENDDATE |where APPNAME!="" and LEVEL="ERROR"|table APPNAME,EVENTTYPE,STARTDATE,ENDDATE,LEVEL,ERRORDESCRIPTION I was getting duplicate entries on result table as below Can anyone please help me with this. Edited: Attached sample json: { "SCMSplunkLog" : { "SCMFailureLog" : { "appName" : "Testing_splunk_alerts_log", "eventType" : "Testing_splunk_alerts_log", "payload" : { "level" : "ERROR", "startTime" : "2022-04-12T13:57:49.156Z", "successCount" : 0, "failureCount" : 0, "publishedCount" : 0, "errorCode" : 0, "errorDescription" : "ERROR: relation \"test.testLand\" does not exist\n Position: 8", "sourceCount" : 0, "endTime" : "2022-04-12T13:57:54.483Z" } } } }  

Hello, I am new to splunk. I am trying to run a report to show what servers our users connect to and on what ports.  the report is going to be reviewed by non IT business unit so it will be nice if the report can have a description to the ports (example: NetBIOS, SQL, LDAP, etc).  Is there a way to do that?  Thank you in advance!

Hello, I am using a scheduled report to fill a summary index. The report is supposed to work with indextime and process everything, that came new within the last hour. Therefore I configured the timerange the following way: But somehow the scheduled report restricts the event time to the last 24 hours, which can yield to no search results in a case, that indexed data is from the day before yesterday: Does someone know, why this restriction is happening, although the event time is supposed to be unrestricted? Thanks and best regards

I have a fairly large(3,400 records) search result that randomly contains non-ascii characters in any one of the 20 fields. This normally is not an issue but the sendemail.py script that is used by splunks email alerting system is erroring out because of it. Is there a way to remove non-ascii characters from all search results? See below for search example:     | inputlookup StatusReport.csv | fields Name ID BusinessGroup Class Email Issues Comments Dynamic Status Owner HoB | sort ID  

We're running Splunk 8.2.2 with the Microsoft Azure Add-on version 3.1.1.  We have the add-on installed on a heavy forwarder. Most of the time the logs pull fine but every 1-2 months they stop pulling.  If we let it sit for ~8 hours it eventually starts working again.  If we disable then re-enable the pull it starts working right away.     What steps should we take next time this happens to troubleshoot why this is occurring? 

Hi, I am looking to upgrade the AppDynamics .Net agent to the latest version on the Windows server during the windows patching activity. There should not be any downtime to restart IIS. So I should configure all the parameters before the restart time. Please provide us with steps to help do this. Regards, Sushma-

Please checkout the idea here (because I don't think currently it's possible with Splunk unless someone has some workaround or solution that I don't know) - https://ideas.splunk.com/ideas/EID-I-1417   (Coping the same content here, recommend upvoting the idea if you think this is currently not possible with Splunk today.) Does anyone know if it is possible to add metadata field(s) to identify all the Splunk instances that have processed a particular event? Let me explain, for example, I'm collecting WinEventLog from instance1 using UF, which is forwarding the logs to an instance2 which is intermediate UF, that is forwarding to intermediate HF (instance3), which is forwarding the data to Indexer (idx1). instance1 (UF) -> instance2 (I UF) -> instance3 (I HF) -> idx1 (Indexer) I want to see if there is a way to get a meta field (indexed time field) that tells the full sequence of where a particular event has traveled through (only Splunk instances of course). This would be useful in complex environment troubleshooting. Even having this as part of debugging we can enable some parameters that can enable this functionality. I don't think currently it's possible unless someone has some workaround or solution.

Have anyone tried sending data from HF to UF? I know it's a stupid question.    And I know it's not going to work. If someone has tried it before intentionally or by mistake, then I'm curious to know what happens in this scenario, what error HF will throw, and what error UF will throw.  

Hello Guys. We use SNMP Modular input to poll data from the devices. We use CISCO, added CISCO MIBs, then added IF-Mib. And after adding IF-MIB file on heavy forwarder in smp_ta folder, on ad hoc search head  we see only 6 fields from IF-MIB (ifDescr_1,... ifDescr_6). When we do snmpwalk from heavy forwarder to the device, we see 87 fields with interfaces names. Seems there will be limits.conf, but maybe you had the  same situation ?

After Running the Query there is no option for Timestamp > Choose Column >  "No result" Kindly see Image.  Since I'm going to use the timestamp column well "Timestamp" currently beginner here