Hi Splunkers, I’m running a Splunk Search Head Cluster (SHC) with 3 search heads, authenticated via Active Directory (AD). We have several custom apps deployed. Currently, users are able to: Create alerts Delete alerts Create reports However, they are unable to delete reports. Investigation Details From the _internal logs, here’s what I observed: When deleting an alert — the deletion works fine: 192.168.0.1 - user [17/May/2025:11:06:59.687 +0000] "DELETE /en-US/splunkd/__raw/servicesNS/username/SOC/saved/searches/test-user-alert?output_mode=json HTTP/1.1" 200 421 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:138.0) Gecko/20100101 Firefox/138.0" - eac203572253a2bd3db35ee0030c6a76 68ms 192.168.0.1 - user [17/May/2025:11:06:59.690 +0000] "DELETE /servicesNS/username/SOC/saved/searches/test-user-alert HTTP/1.1" 200 421 "-" "Splunk/9.4.1 (Linux 6.8.0-57-generic; arch=x86_64)" - - - 65ms   When deleting a report — it fails with a 404 Not Found: 192.168.0.1 - user [17/May/2025:10:27:51.699 +0000] "DELETE /en-US/splunkd/__raw/servicesNS/nobody/SOC/saved/searches/test-user-report?output_mode=json HTTP/1.1" 404 84 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:138.0) Gecko/20100101 Firefox/138.0" - eac203572253a2bd3db35ee0030c6a76 5ms 192.168.0.1 - user [17/May/2025:10:27:51.702 +0000] "DELETE /servicesNS/nobody/SOC/saved/searches/test-user-report HTTP/1.1" 404 84 "-" "Splunk/9.4.1 (Linux 6.8.0-57-generic; arch=x86_64)" - - - 1ms   Alerts are created under the user’s namespace (servicesNS/username/...) and can be deleted by the user. Reports appear to be created under the nobody namespace (servicesNS/nobody/...), which may be the reason users lack permission to delete them. Has anyone faced a similar issue? ... View more

Hi everyone, I have 3 indexers (in a cluster) located on Site A. The current replication factor (RF) is set to 3. I need to move operations to Site B. However, for technical reasons, I cannot physically move the existing data — I will simply have 3 new indexers at Site B. Here’s the plan I’m considering: Launch 1 new indexer at Site B. Add the new indexer to the existing cluster. Increase the RF to 4 (so that all raw data is fully replicated across the available indexers). Shut down all 3 indexers at Site A. Decrease the RF back to 3. (I understand there is a risk of some data loss.) Add 2 additional new indexers to the cluster at Site B. My main concern is step 5 — decreasing the RF — which I know is not best practice, but given my situation, I don't have many options. Has anyone encountered a similar situation before? I'd appreciate any advice, lessons learned, or other options I might not be aware of. Thanks in advance! ... View more

Hi Splunkers! I have a problem with props.conf and tranforms.conf I face with this error in Linux Servers.   multipathd[212317]: sdb: failed to get sgio uid: No such file or directory multipathd[212317]: sdb: add missing path   So I set props.conf and transforms.conf to get rid of this messages. It seems correct and I can't figure out why this doesn't work?!   props.conf [syslog] TRANSFORMS-null = setnull     transorms.conf [setnull] REGEX = multipathd DEST_KEY = queue FORMAT = nullQueue   I also tried these REGEXs   REGEX = .*multipathd.* REGEX = (.+multipathd.+)   But nothing happened So where did I make a mistake? appreciate for your time ... View more

Hi Splunkers! I have a problem with line breaking in Splunk add-on F5-bigip. I've tried some regex to break the line correctly but I'm not successful. First of all for simplicity I changed my outputs.conf in Heavy Forwarder. outputs.conf   [indexAndForward] index = true     In fact the   indexing is false on this node and this HF forward data to my indexer cluster and I also have search head cluster. But as I mentioned just for simplicity I turned mu indexing to true in this HF. Then I used these regexes to break the lines   props.conf   [f5:bigip:syslog] # LINE_BREAKER = ^()\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2} # LINE_BREAKER = ^\w{3}\s\d*\s\d{2}\W\d{2}\W\d{2} LINE_BREAKER = ([\r\n]+)\w{3}\s\d+\s\d{2}\W\d{2}\W\d{2} # LINE_BREAKER = ([\r\n]+) # LINE_BREAKER = \n MAX_TIMESTAMP_LOOKAHEAD = 16 # ADD_EXTRA_TIME_FIELDS = subseconds NO_BINARY_CHECK = true # EVENT_BREAKER_ENABLE = false # TIME_FORMAT = %b %d %H:%M:%S TIME_PREFIX = SHOULD_LINEMERGE = false TRUNCATE = 1000000     This is some of my data that I can't break the line correctly.     Sep 18 19:12:27 192.168.1.1 Sep 18 14:42:27 F5-LTM-3.company.local info logger[25169]: [ssl_req][18/Sep/2021:14:42:27 +0000] 1.1.1.1 TLSv1.2 ECDHE-RSA-AES128-SHA "/mgmt/shared/inflate/available" 2 Sep 18 19:12:28 192.168.1.1 Sep 18 14:42:28 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673914804247",request_status="alerted",response_code="302",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTP",query_string="",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:28",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="44180",dest_port="80",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTP",uri="/account/login",fragment="",request="GET /Account/Login HTTP/1.1\r\nConnection: keep-alive\r\nHost: example.com\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-US,en;q=0.8,en-US;q=0.5,en;q=0.3\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/5.0 (Android 7.0; Mobile; rv:68.0) Gecko/68.0 FUSefox/68.0\r\nUpgrade-Insecure-Requests: 1\r\nX-Forwarded-For: 1.1.1.1\r\n\r\n",response="Response logging disabled" Sep 18 19:12:28 192.168.1.1 Sep 18 14:42:28 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673951684370",request_status="alerted",response_code="302",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTP",query_string="",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:28",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="19338",dest_port="80",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTP",uri="/account/login",fragment="",request="GET //Account/Login HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 FUSefox/92.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: keep-alive\r\nCookie: GuidedTourVersion=1; SiteVersion=3.7.6; __utma=226054936.2062308401.1625890970.1631960683.1631966584.238; __utmz=226054936.1625890970.1.1.utmcsr=(dUSect)|utmccn=(dUSect)|utmcmd=(none); crisp-client%2Fsession%2Fbb1636a8-4b45-4fbb-971e-d5e50e2a1e1f=session_230233c6-895e-42d0-b257-4ae4c1903150; _hjid=b846f33d-e2e6-4c9a-a757-f9ab405b0193; Token=6abe8980-5856-4d6f-b05a-2915b970983e; lastmessage-6=87696; lastmessage-4=1; lastmessage-2=undefined; text0_1567617252=true; text0_496056564=true; .ASPXAUTH=4A5473E3674D47ED86E8EA52D6A4613C2F30F1D31A41DF7F8BEDBAB120DE5ACEB8E3DD46D71 Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673926887289",request_status="alerted",response_code="302",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTP",query_string="",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="46453",dest_port="80",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTP",uri="/account/login",fragment="",request="GET //Account/login HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Linux; Android 7.1.1; SM-J510F Build/NMF26X; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/64.0.3282.137 Mobile Saenri/537.36 AgentWeb/4.1.3 UCBrowser/1.1.1.1\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en,en-US;q=0.9,en-GB;q=0.8,en-US;q=0.7\r\nX-Requested-With: com.sefryekcompany.mobiletradingpro\r\nX-Forwarded-For: 1.1.1.1\r\n\r\n",response="Response logging disabled" Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673919202912",request_status="alerted",response_code="301",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTPS",query_string="37419741",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="f8689163755118a6",src_port="44760",dest_port="443",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared1-HTTPS",uri="/serviceworker.js",fragment="",request="GET /serviceworker.js?37419741 HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\nAccept: */*\r\nSave-Data: on\r\nService-Worker: script\r\nSec-Fetch-Site: same-origin\r\nSec-Fetch-Mode: same-origin\r\nSec-Fetch-Dest: serviceworker\r\nReferer: https://mobile.bmibourse.com/serviceworker.js?37419741\r\nUser-Agent: Mozilla/5.0 (Linux; Android 10; SM-A207F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Mobile Saenri/537.36\r\nAccept-Encoding: gzip, deflate, br\r\nAccept-Language: en-US,en;q=0.9,en-US;q=0.8,en;q=0.7\r\nCookie: basket-warning-readed=1; basket-option-visited=true; tag-market-map-visited=true; index-technical-visited=true; stock-technical-visited=true; AppVersion=1.1.2; TS01e42c80=0180bb6f222b77a4b3dd30e3eddfc570acb1a0674cc23f80304088a610b57e5e43c686eb7415c18bc949724b74a1f77b7746en6cd8\r\nX-Forwarded-For: 5.116.208 Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673963109971",request_status="alerted",response_code="301",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTPS",query_string="37418741",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="80b2664635b96eeb",src_port="41628",dest_port="443",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared3-HTTPS",uri="/serviceworker.js",fragment="",request="GET /serviceworker.js?37418741 HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Android 7.0; Mobile; rv:68.0) Gecko/68.0 FUSefox/68.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.8,en-US;q=0.5,en;q=0.3\r\nAccept-Encoding: gzip, deflate, br\r\nService-Worker: script\r\nConnection: keep-alive\r\nCookie: _ga=GA1.2.1098137509.1594471619; basket-warning-readed=1; basket-option-visited=true; AppVersion=1.1.2; index-technical-visited=true; tag-market-map-visited=true\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nX-Forwarded-For: 1.1.1.1\r\nSSLcompany: 1\r\n\r\n",response="Response logging disabled" Sep 18 19:12:29 192.168.1.1 Sep 18 14:42:29 F5-LTM-3.company.local ASM:unit_hostname="F5-LTM-3.company.local",management_ip_address="192.168.1.1",management_ip_address_2="",http_class_name="/Common/Online",web_application_name="/Common/Online",policy_name="/Common/Online",policy_apply_date="2021-08-26 10:13:52",violations="Illegal redUSection attempt",support_id="13616148673952377578",request_status="alerted",response_code="301",ip_client="1.1.1.1",route_domain="0",method="GET",protocol="HTTPS",query_string="37418741",x_forwarded_for_header_value="1.1.1.1",sig_ids="",sig_names="",date_time="2021-09-18 14:42:29",severity="Error",attack_type="Other Application Activity",geo_location="US",ip_address_intelligence="N/A",username="N/A",session_id="3915b37e523c6d41",src_port="55434",dest_port="443",dest_ip="1.1.1.1",sub_violations="",vUSus_name="N/A",violation_rating="3",websocket_dUSection="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="",staged_threat_campaign_names="",blocking_exception_reason="N/A",captcha_result="not_received",microservice="",vs_name="/Common/OnlineShared2-HTTPS",uri="/serviceworker.js",fragment="",request="GET /serviceworker.js?37418741 HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\nAccept: */*\r\nService-Worker: script\r\nX-Requested-With: com.sefryekcompany.mobiletradingpro\r\nSec-Fetch-Site: same-origin\r\nSec-Fetch-Mode: same-origin\r\nSec-Fetch-Dest: serviceworker\r\nReferer: https://mobile.mobinsb.com/serviceworker.js?37418741\r\nUser-Agent: Mozilla/5.0 (Linux; Android 10; SM-A600G Build/QP1A.190711.020; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/81.0.4044.138 Mobile Saenri/537.36\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-US,en;q=0.9,en-CA;q=0.8,en-US;q=0.7,en;q=0.6\r\nCookie: companyRLCUrl=////////////////////////////////////core.companyrlc.com/; companyRLApiUrl=//rlcchartapi.companyrlc.com/; BrokerId=777; ThemeName=MobinSarmayeh; DisabledModules=changebroker; PushSubDomainName=push2v7.company.co      Thanks in advance ... View more

Hi Splunkers   I've tried to read some data from MS SQL Server. The data is json like. It works for a while and then I encounter with this message:   ERROR HttpInputDataHandler - Failed processing http input, token name=db-connect-http-input, channel=n/a, source_IP=127.0.0.1, reply=6, events_processed=802, http_input_body_size=11904838 ERROR HttpInputDataHandler - Parsing error : While expecting event's raw text: String value too long. valueSize=5246755, maxValueSize=5242880, totalRequestSize=11904838     After that no data getting in. Is there any way to increase the maxValueSize? or my problem is originated from elsewhere Thanks in advance ... View more