I have this kind of weird custom app (and dangerous too) that changes the UF Instance GUID.  Basically, I created a .sh file, which utilizes "sed" command on Linux, to change the UUID value of the /opt/splunkforwarder/etc/instance.cfg file. To use a .sh script and make changes to SPLUNK_HOME directory is quite a dangerous task, I advised not to, however, this task is quite simple, I tested so I decided to deploy an app called REGEN_GUID with a single inputs.conf file that have the stanza to run the script. [script://./bin/regenerate_guid.sh] interval = 900 source = regenerate_guid sourcetype = regenerate_guid index = <REDACTED> disabled = 0 In general, quite simple, and it run. I could change the instance UUID and nothing critical happened. However, of course after I see that the UUID has been changed, I would remove the client from the app. I used the deployment server UI, go into the app section and remove the IP of the instance from the whitelist. Checking the splunkd.log, I could see the log when it say it is removing the app However, after that, I check again and see the log and see it is still finding the script to run, the log appear every 15 minutes, which is equal to the script interval, so basically the UF is still interpreting the task of running the script. The log is like this: 05-07-2025 11:00:07.938 +0700 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/REGEN_GUID/bin/regenerate_guid.sh" /bin/sh: 1: /opt/splunkforwarder/etc/apps/REGEN_GUID/bin/regenerate_guid.sh: not found Does anyone know the reason? I think the reason might be the way Splunk monitor script inputs is through some kinds of cron file, and my app failed to update that when it was removed? ... View more

I have this kind of log: Mar 18 02:32:19 MachineName python3[948]: DEBUG:root:... Dispatching: {'id': '<id>', 'type': 'threat-detection', 'entity': 'threat', 'origin': '<redacted>', 'nature': 'system', 'user': 'system', 'timestamp': '2025-03-17T19:32:17.974Z', 'threat': {'id': '<redacted_uuid>', 'maGuid': '<redacted_guid>', 'detectionDate': '2025-03-17T19:32:17.974Z', 'eventType': 'Threat Detection Summary', 'threatType': 'non-pe-file', 'threatAttrs': {'name': '<filename>.ps1', 'path': 'C:\\Powershell\\Report\\<filename>.ps1', 'md5': '<redacted_hash>', 'sha1': '<redacted_hash>', 'sha256': '<redacted_hash>'}, 'interpreterFileAttrs': {'name': 'powershell.exe', 'path': 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe', 'md5': '097CE5761C89434367598B34FE32893B', 'sha1': '044A0CF1F6BC478A7172BF207EEF1E201A18BA02', 'sha256': 'BA4038FD20E474C047BE8AAD5BFACDB1BFC1DDBE12F803F473B7918D8D819436'}, 'severity': 's1', 'rank': '100', 'score': '50', 'detectionTags': ['@ATA.Discovery', '@ATA.Execution', '@ATE.T1083', '@ATE.T1059.001', '@MSI._apt_file_psgetfiles', '@ATA.CommandAndControl', '@ATE.T1102.003', '@MSI._process_PS_public_repos', '@MSI._process_ps_getchilditem', '@ATE.T1105', '@ATE.T1071.001', '@MSI._process_pswebrequest_remotecopy', '@ATA.DefenseEvasion', '@ATE.T1112', '@MSI._reg_ep0029_intranet'], 'contentVersion': None}, 'firstDetected': '2025-03-17T19:32:17.974Z', 'lastDetected': '2025-03-17T19:32:17.974Z', 'tenant-id': '<redacted_tenant_id>', 'transaction-id': '<redacted_transaction_id>'} The "Dispatching" I want it to be a required text, so only log that have this keywork would I apply transforming. I want to parse the JSON part so I can use its fields, like json_data.threatAttrs.name. Any suggestions? I tried the regex editor UI, but it broke down since it couldn't differentiate the "name" fields, since the same field name appeared. So I am thinking of using props.conf and transforms.conf, but I don't know how. Any help would be appreciated! ... View more

I have a multisite setup. Each site has 3-4 indexers, with a Replication Factor = 2. Search Factor is = 1. When querying for data from all the sites, how to determine which site is taking the longest to search for data? ... View more