I have two events where in order to get a response time, I need to subtract the two timestamps. However, this needs to be grouped by "a_session_id" / "transaction_id." The two events I need are circled in red in the screenshot attached. I need those two events out of the three events. Every "a_session_id" has these three logs. source="/apps/logs/event-aggregator/gateway_aggregator_events.log" is always after source="/logs/apigee/edge-message-processor/messagelogging/gateway-prod/production/Common-Log-V1/14/log_message/gateway.json" Please let me know if you need more information. Such as snippets on the SPL. Any assistance is much appreciated!

Hello I try to summarize the different steps to onboard automatically a csv file in Splunk 1) On the forwarder: - I need an inputs.conf to tell the forwarder what data to send. (And eventually props.conf) http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf - I also need an outputs.conf to tell the forwarder where to send the data.  inputs.conf [monitor://C:\Program Files\SysCheck\Logs\*.txt] outputs.conf [tcpout:anyName] server=indexer.myco.com:9997 http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf   2) On the indexer I need to configure the receiving port on http://docs.splunk.com/Documentation/Splunk/6.2.1/Forwarding/Enableareceiver Is it correct? Thanks

Hello, how to remove and clean corrupt peer (indexer) from cluster? Should we stop it then after hardware maintenance delete all indexes data directories then start it again to resync whole data from other peers? Thanks!  

I am trying to UPGRADE using Ansible, I kick off the playbook via the bastion host. Here are the tasks. 1. copy the install file to remote 2. stop the Splunk service 3. install Splunk Forwarder 9.1 4. reboot 5. start the Splunk service All are fine until step 4. I ssh to the specific host and checked the status. It was not running. I scratched my head and tried something like below. sudo to root /opt/splunkforwarder/bin/splunk version it prompted for the license & perform upgrade message. I typed Y for both options. after a few minutes (it showed a message to disable boot start), returned to prompt. disabled boot start reboot sudo systemctl start splunk Finally, it's up & running.  How do I fix step 5. I have 100s of ec2 instances to upgrade.  

I have a field with data like this: loggingObject.methodName="WXYX.MNOController.myMethodName". loggingObject.methodName="DEF.GHI.TUVController.myMethodName2"   I want to extract just the myMethodName part.  If the dot before it is there, that is fine. I tried using the reg ex field extractor, this is what it came up with: ^(?:[^\.\n]*\.){9}(?P<methodName>\w+) But it seems like it's creating a name for the extracted field, "methodName". I then tried to use it my query like this:   | regex methodName="^(?:[^\.\n]*\.){9}(?P<methodName>\w+)"   But it doesn't work.  There also isn't anything in that line that tells it to extract from the loggingObject.methodName field specifically. How can I extract what I'm trying to extract?

Hi,  I am trying to show successful validations and failures in one of the dashboard panels. I am logging exceptions in traceData.exception field. In this field there is exception with full stacktrace , but would like to display just exception name so need to extract just first line of exception. My query looks like index=xxx sourcetype="xxx" app=xxx event.data.request.uri="/xxx" | rename event.data.response.statusCode as statusCode | rename event.traceData.exception as exception | rex field=exception "(?<exception>.*)" | eval result = if(statusCode=201,"Valid", 'exception') | timechart span=1h count by result​ the issue is it displays "Valid" calls, but for exceptions it just displays one exception, and the other one is NULL.    _time Valid Token invalid  NULL 2022-10-13 08:00 1 1 1   Both exceptions have the same fields (just different exception values and stacktrace). Could you help me with the query which will display results and extract all the exceptions without stacktrace ?

Hi All, I'm trying to optimize the following search because it runs very slow.  Looking for some help w/it.  I've been exploring the multi-search command also, but cannot figure out how to get it working yet... | union [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=ShowDeviceProgramOffer AND completion_code=0) | join type=inner session [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=ShowBackupDeviceProgramOffer AND completed_from_ui=False AND completion_code=0)] | join type=inner session [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=SetUserOnboardingComplete AND completion_code=0)] | timechart span=5m dc(session) as total1] [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=ShowDeviceProgramOffer AND completion_code=0) | join type=inner session [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=ShowBackupDeviceProgramOffer AND completed_from_ui=False AND completion_code=0)] | join type=inner session [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=SetUserOnboardingComplete AND completion_code=0)] | join type=inner session [search index=xmo2-aws host=xmo2-prod* sourcetype=oss_app_log "Step completion_code set/reset" (flow_name=UcdeDeviceOnboarding AND step_name=ShowUcdeActivationStatus AND completion_code=0)] | timechart span=5m dc(session) as total2] | timechart span=5m sum(total1) as eval1, sum(total2) as eval2 | eval ActivationFailed=eval1-eval2 | timechart span=5m sum(eval2) as "Accepted & SignedIn", sum(ActivationFailed) as "Activation Failed" partial=f

Is there a recommended way to update Splunk apps in clustered environments? Based on some app instructions, the recommended approach is to copy over the app archive contents into /etc/shcluster/apps/ (or /etc/manager-apps/ for CM). This overwrites existing contents and should preserve the local directory (unless the upgraded app has a local directory). Should I follow that for all apps? Same question for standalone servers: should I use the above approach or use the install CLI command?

The CIM documentation says that we should install CIM only on SH. But it contains an indexes.conf in default. Should we leave the indexes.conf in the SHC? in this case the index defined inside indexes.conf wont be usable because is not defined in the indexer cluster. We dont know if it is correct to define the CIM indexes.conf in the SHC instead of the indexers. Anyone managed to install CIM in a clustered environment without issues?