About ww9rivers

ww9rivers · ‎11-01-2024

This is what I have in "server.conf", in addition to what I have in "web.conf": [httpServer] disableDefaultPort = false mgmtMode = tcp After that, splunkd starts to listen to TCP port 8089.

ww9rivers · ‎09-12-2024

I see this "extracted_eventtype" field in many saved searches and dashboard inline searches. However, I cannot find where it is generated. In the DUO events I do see "event_type" and "eventtype" fields. But not "extracted_eventtype". Dashboards with that field show "No results found." because that field is nowhere to be found in DUO events. Any thoughts / pointers would be very much appreciated!

ww9rivers · ‎06-24-2024

From the documentation, I believe that the Task Server should start after I set up the JAVA_HOME, but it has been failing to start, with only a message "Failed to restart task server." I am running an instance of Splunk version 8.1.5. When installing DBx 3.17.2, I installed OpenJDK 8, DBx started that it required Java 11. So I installed java-11-openjdk version 11.0.23.0.9. Task Server JVM Options were automatically set to "-Ddw.server.applicationConnectors[0].port=9998". Is there anything else missing? Is there a way to debug this issue? I looked into the internal logs from this host but have not been able to find anything that stands out. Thanks for any insights and thoughts.

ww9rivers · ‎06-19-2024

Thank you, @akouki_splunk! That's it. The on-prem Splunk instance uses SAML authentication so I get automatically assigned both "admin" and "user" roles from my group memberships. The "user" role was in the "blacklisted_roles" list, which caused the error. Thank you for the quick response!

ww9rivers · ‎06-19-2024

Sorry for missing the details ... The message came from the app, not Splunk itself. Splunk itself is a standalone instance, version 8.1.5, running on a RHEL 8.10 Linux VM. I downloaded the package from Splunkbase and installed it with "install app from file". Thank you for taking the time!

ww9rivers · ‎06-18-2024

The issue is actually with the "savedsearch" command. The document states: When the savedsearch command runs a saved search, the command always applies the permissions associated with the role of the person running the savedsearch command to the search. The savedsearch command never applies the permissions associated with the role of the person who created and owns the search to the search. This happens even when a saved search has been set up to run as the report owner. That makes me wonder, however, when is the "run as owner" setting actually applies the owner's permission when the search is run? Can another user embed the search in a dashboard? Probably not with the "savedsearch" command either. So what is the use case for that feature?

ww9rivers · ‎06-18-2024

Under that subject line, the detail says: You do not have necessary authorization to access and use this application : App Content Manager. Access to all of its features has been restricted. If you believe this is in error, or if you require access for a specific reason, please reach out to your Splunk administrator for further assistance. But I am the Splunk Admin. This app is quite new and not supported by Splunk. So I am trying to get the authors' insights or anyone who has experiences with it. Much appreciated!

ww9rivers · ‎05-27-2024

Based on documentation, and posts (Who do saved scheduled searches run as? and Question about "run as" (Owner or User ) for saved searches), a saved search configured to "run as" owner, should run with permissions that the owner of the search has. However, I have two saved searches that do not work that way. Specifically, the searches use indexes that I (the owner) has access to but other user roles do not. The difference that I can think of is that my searches are in a Splunk Cloud instance, and my users authenticate using SAML against a IdP on premise. Any insights would be much appreciated!

ww9rivers · ‎04-22-2024

I am having the same issue. I have also checked the Release Notes you linked. I already have those items configured: $ bin/splunk btool web list | grep mgmtHostPort mgmtHostPort = 0.0.0.0:8089 $ bin/splunk btool server list | grep disableDefaultPort disableDefaultPort = false But still, I don't see splunkd listening on port 8089: $ sudo lsof -i tcp -P | grep 8089 (I get nothing.) The Universal Forwarder is v9.2.1 on Red Hat Enterprise Linux 8.9.

ww9rivers · ‎11-20-2023

Our networking team needs to get ASN from public IP addresses. We found the TA-asngen add-on. I put it through splunk-inspect and fixed the failures, added a MaxMind license key in the default/asngen.conf, installed it in our Splunk Cloud instance. When I try to run the `asngen` command, it gives out this error message: Exception at "/opt/splunk/etc/apps/TA-asngen/bin/asngen.py", line 55 : maxmind license_key is required Just wonder if anyone has tried the TA in the cloud. Any thoughts would be much appreciated.

ww9rivers · ‎08-30-2023

Splunk seems to have a problem with authenticating a SAML user account using a token. The purpose of using token authentication is to allow an external application to run a search and get the results. A sample script is posted on GitHub as a code gist — the script simply starts a search but does not wait for the results. The problem is that when token authentication is used with a SAML account, it only works when that SAML user is logged in on the Splunk web GUI and while the interactive session is (still) valid. The problem is shown in the internal log: 07-03-2023 19:35:53.931 +0000 ERROR Saml [795668 AttrQueryRequestExecutorWorker-0] - No status code found in SamlResponse, Not a valid status. 07-03-2023 19:35:53.901 +0000 ERROR Saml [795669 AttrQueryRequestExecutorWorker-1] - No status code found in SamlResponse, Not a valid status. The theory on the failure is: The token authentication works with (within) Splunk; But Splunk needs to perform RBAC after authentication. So it does AQR after the authentication; However, when there is no valid, live SAML session, the AQR fails. (AQR = Attribute Query Request) -- in this case, to get the user's group memberships to map to Splunk roles. I wonder if anyone has been able to get token authentication to work for a SAML account? [Edit]: On the other hand, is it simply impossible to use token authentication with a SAML user account?

ww9rivers · ‎08-20-2023

Got this figured out! The JS version sent the `body` part wrong: It is not supposed to be JSON encoded but HTTP query string encoded. The working version is here in GitHub: https://gist.github.com/ww9rivers/dc3fd9ba8d2817b9fc986aa9457a2b61

ww9rivers · ‎08-10-2023

There is something missing in my NodeJS code, it seems. This simple Python3 test works (in creating a search job and returning an sid): import os import requests # Set up the session with our adapter SEARCH_ENDPOINT = "https://"+os.environ['SPLUNK_HOST']+":8089/services/search/jobs" headers = { 'Authorization': 'Bearer '+os.environ['SPLUNK_TOKEN'], "Accept": "application/json" } params = { "search": "inputcsv search-output.csv", "output_mode": "json" } response = requests.post(SEARCH_ENDPOINT, data=params, headers=headers, verify=True) print(response.text) But this NodeJS code does not: const SEARCH_ENDPOINT = `https://${process.env.SPLUNK_HOST}:8089/services/search/jobs`; const data = { search: "inputcsv search-output.csv", output_mode: "json" }; const options = { method: "POST", mode: "cors", cache: "no-cache", credentials: "same-origin", headers: { Authorization: `Bearer ${process.env.SPLUNK_TOKEN}`, Accept: "application/json" }, redirect: "follow", referrerPolicy: "no-referrer", body: JSON.stringify(data), }; let response = await fetch(SEARCH_ENDPOINT, options); console.log(response.status); console.log(response.body); console.log(await response.json()); With the same SPLUNK_HOST and SPLUNK_TOKEN values, the Python code produces an output like this: {"sid":"1691684765.268000"} But the NodeJS example returns an XML document. Any thoughts are much appreciated!

ww9rivers · ‎07-21-2023

No. Actually, in the answer that you linked, you clearly used "/services/search/jobs/" to create the search: curl -ku <user:pass> https://localhost:8089/services/search/jobs/ -d search=. . . In my case, I am trying to use the same API endpoint to create a search. My search command is not necessarily a "|rest" , rather, it is something like "| inputcsv <some-results>.csv" for most my use cases. Thank you for the thoughts.

ww9rivers · ‎07-20-2023

Following the documentation here: https://docs.splunk.com/Documentation/Splunk/latest/RESTTUT/RESTsearches#Create_a_search_job I expect that a successful REST API call to endpoint "/services/search/jobs" would return a single job ID as the document shows. However, in my testing, when the call returns with a status of 200 (success), the response data contains an object, which contains 6 keys: Object.keys(jobId) = (6) ['links', 'origin', 'updated', 'generator', 'entry', 'paging'] where, jobId.entry is an array of hundreds of search jobs -- basically the call to create a search job returned a list of all the jobs in the search head. The code (JavaScript) is in this public repository: https://github.com/ww9rivers/splunk-rest-search Am I missing anything? Thank you for your insights!

ww9rivers · ‎11-29-2022

Got these errors while installing acs-cli in WSL/Ubuntu: ==> Installing acs from splunk/tap ==> Installing dependencies for splunk/tap/acs: [email protected], glibc, gmp, isl, mpfr, libmpc, lz4, xz, zlib, zstd, binutils and gcc ==> Installing splunk/tap/acs dependency: [email protected] ==> Pouring [email protected]_64_linux.bottle.1.tar.gz cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_connmark.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_dscp.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_mark.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_rateest.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_tcpmss.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter_ipv4/ipt_ecn.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter_ipv4/ipt_ttl.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter_ipv6/ip6t_hl.h': File exists Error: Failure while executing; `cp -pR /tmp/d20221130-6079-1ef476r/[email protected]/. /home/linuxbrew/.linuxbrew/Cellar/[email protected]` exited with 1. Here's the output: cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_connmark.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_dscp.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_mark.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_rateest.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter/xt_tcpmss.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter_ipv4/ipt_ecn.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter_ipv4/ipt_ttl.h': File exists cp: cannot create regular file '/home/linuxbrew/.linuxbrew/Cellar/[email protected]/5.15.57/include/linux/netfilter_ipv6/ip6t_hl.h': File exists This happened after the "brew install acs" command has gone through 13 downloads of libraries etc. Any help would be much appreciated.

ww9rivers · ‎11-27-2022

In my case, all the 6 dashboards currently in the local/ folder all exist in the deployer, so they have been deployed at some point. Why only two of them have the "Move" and "Delete" options? I guess my ultimate question is: What makes the search head cluster nodes decide which dashboards can be deleted and not the others?

ww9rivers · ‎11-24-2022

I did not check that. I just checked now. All the six dashboards are found on the deployer, also in the local/ folder (I guess it is desired for this app to always be editable by the security team.) -- Furthermore, there are more dashboards on the deployer. The team is cleaning up the app. It seems that they have been able to delete most of what they want to delete. By the way, I have also checked and double checked the permissions on all the dashboards. They all have "write" permission granted to the admin role.

ww9rivers · ‎11-24-2022

This seems to be a bit strange: We are running Enterprise version 8.1.5 in a search head cluster. A custom app is created for our security team to manage their dashboards, etc. The strange thing is, some of the dashboards cannot be deleted -- there is just no delete (or move) option: I've checked on all the individual nodes in the cluster, they are all in the local/ folder in the app. This is on-prem Splunk Enterprise, so I can manually delete them from all the nodes as an admin. But I would like to understand what I am missing here. I did search for answers here and found this one post. But that is in the Splunk Cloud. So I am not sure if my issue is a bug like that in the posting or not. Thanks for any thoughts / discussions. Happy holidays!

ww9rivers · ‎08-31-2022

Thank you @burwell and @isoutamo for your responses. I believe you are right about the upgrade path. We spend a bit more time on this. There is actually some logging in web_service.log file that provided hints about the problem. The error is in the TA-SMTP-Reputation add-on in our particular case. The TA is probably not compatible with Python 3. After removing the TA from the apps folder, the server starts OK now.

ww9rivers · ‎08-31-2022

Try this: index=test1 sourcetype=abc | table detail.trackInformation.correlationId Obviously, you can rename that field if you want to.

ww9rivers · ‎08-30-2022

I just upgraded a dev instance from 7.3.4 to 9.0.1, and splunkd would start but the web UI stopped working. Found these in splunkd.log: 08-30-2022 12:43:16.300 -0400 ERROR UiPythonFallback [22665 WebuiStartup] - Couldn't start appserver process on port 8065: Appserver at http://127.0.0.1:8065 never started up. Set `appServerProcessLogStderr` to "true" under [settings] in web.conf. Restart, try the operation again, and review splunkd.log for any messages that contain "UiAppServer - From appserver" 08-30-2022 12:43:16.300 -0400 ERROR UiPythonFallback [22665 WebuiStartup] - Couldn't start any appserver processes, UI will probably not function correctly! 08-30-2022 12:43:16.300 -0400 ERROR UiHttpListener [22665 WebuiStartup] - No app server is running, stop initializing http server However, after adding the "appServerProcessLogStderr = true" setting to web.conf, I only see this one line in splunkd.log: 08-30-2022 12:48:53.628 -0400 INFO UiAppServer [28199 appserver-stderr] - Starting stderr collecting thread No message with "UiAppServer" after that. Any thoughts / help would be much appreciated!

ww9rivers · ‎01-06-2022

I am having the same issue with version 8.1.5. We upgraded from 7.3.4 a few months ago but I don't know if this has happened before. The error message is the same: "Unable to access online documentation. Please ensure docsCheckerBaseURL in web.conf is correct." I get that with any inline help or Help >> Tutorials. So the URL is: https://localhost:8000/en-US/help?location=search_app.tutorial I looked in web.conf using Splunk btool, which is the default setting: app/splunk/etc/system/default/web.conf docsCheckerBaseURL = https://quickdraw.splunk.com/help When I manually put that URL in the browser, I get a complete blank response with status 200. I checked the Chrome browser console in the developer tool. There is no error about client blocking the request. The error messages is in the HTML response. What else can I check? Thanks!

ww9rivers · ‎12-08-2021

1) As I stated in my first post, I reported that as a bug to Splunk Support. But the support engineer insists that it is not a bug, but a documentation error. I have not been able to convince him/her in any way. 2) The documentation is NOT incorrect as far as reflecting what the Splunk core recognizes as a correct keyword -- note that this stanza works for all other apps as well, not just this app.

ww9rivers · ‎12-07-2021

@richgalloway wrote: You should submit feedback on the docs pages so the Documentation team is aware of the discrepancy and can correct it (they're good about that). But it is NOT a documentation error. Splunk 8.1.5 works with [replicationBlacklist], as the document correctly states. It does NOT work with [replicationDenylist]. Not sure what you mean by "the error in your local distsearch.conf file". I did not have a local/distsearch.conf file. I had to create a local/distsearch.conf file to work around this bug. That is exactly what I meant. Sorry, I do not understand what exactly you mean.

Posts	124
Solutions	7
Karma Given	29
Karma Received	22
Member Since	‎07-27-2011

Online Status	Offline
Date Last Visited	‎10-03-2025 07:16 PM

Role-based search filter syntax

Splunk Enterprise 10.0.0 unable to "browse more ap...

CiscoSecurityCloud app not parsing fields for cisc...

Data missing from universal forwarder on a Linux h...

"Pipeline data does not have indexKey" in a simple...

Where is the extracted_eventtype field in the duo_...

Splunk DB connect 3.17.2 fail to start Task Server...

Got error after installation: OOOOPS - UNAUTHORIZE...

"Run as" owner not seem to work for saved searches

TA-asngen for Splunk Cloud?

Re: Splunk Universal forwarder management port is ...

Where is the extracted_eventtype field in the duo_...

Splunk DB connect 3.17.2 fail to start Task Server...

Re: Got error after installation: OOOOPS - UNAUTHO...

Re: Got error after installation: OOOOPS - UNAUTHO...

Re: "Run as" owner not seem to work for saved sear...

Got error after installation: OOOOPS - UNAUTHORIZE...

"Run as" owner not seem to work for saved searches

Re: Splunk Universal forwarder management port is ...

TA-asngen for Splunk Cloud?

Does Splunk have a problem with token authenticati...

Re: Creating a search job using the REST API

Re: Creating a search job using the REST API

Re: Creating a search job using the REST API

How can I create a search job using the REST API?

Error installing acs-cli using linuxbrew

Re: Unable to delete dashboards in an app

Re: Unable to delete dashboards in an app

Unable to delete dashboards in an app

Re: 7.3.4 to 9.0.1 upgrade - WARNING: web interfac...

Re: How to extract a field from raw json?

[SOLVED] 7.3.4 to 9.0.1 upgrade - WARNING: web int...

Re: Unable to access online documentation

Re: Bug in Splunk_SA_Scientific_Python_linux_x86_6...

Re: Bug in Splunk_SA_Scientific_Python_linux_x86_6...

Join the Conversation