Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- About aaronbarry73
aaronbarry73
Path Finder
Member since:
05-31-2018
05-19-2024
Community Statistics
| Posts | 14 |
| Solutions | 4 |
| Karma Given | 6 |
| Karma Received | 6 |
| Member Since | 05-31-2018 |
Activity Feed
- Got Karma for Re: DB Connect 3.1.4 not working. 01-29-2025 03:43 PM
- Posted Re: DB inputs only running on Search Head Cluster Captain on Deployment Architecture. 12-19-2023 12:09 PM
- Karma Re: DB inputs only running on Search Head Cluster Captain for gcusello. 12-14-2023 08:30 AM
- Posted Re: DB inputs only running on Search Head Cluster Captain on Deployment Architecture. 12-14-2023 08:26 AM
- Posted DB inputs only running on Search Head Cluster Captain on Deployment Architecture. 12-11-2023 04:02 PM
- Karma Re: Flatten JSON associative array with variable key names that even containing dots for spavin. 08-23-2023 05:37 AM
- Posted Re: DB Connect 3.1.4 not working on All Apps and Add-ons. 08-07-2023 08:21 AM
- Karma Re: DB Connect 3.1.4 not working for edhealea. 08-07-2023 08:17 AM
- Karma Re: DBConnect: OSError: [Errno 2] No such file or directory validate java command: /usr/java/latest/bin/java. for teunlaan. 06-05-2020 12:50 AM
- Got Karma for Re: How do we change the default search period to one hour?. 06-05-2020 12:50 AM
- Got Karma for Re: How do we change the default search period to one hour?. 06-05-2020 12:50 AM
- Got Karma for DB Connect 3: Why am I getting error "failed to fetch" when I edit or clone a DB Input?. 06-05-2020 12:49 AM
- Got Karma for DB Connect 3: Why am I getting error "failed to fetch" when I edit or clone a DB Input?. 06-05-2020 12:49 AM
- Got Karma for DB Connect 3: Why am I getting error "failed to fetch" when I edit or clone a DB Input?. 06-05-2020 12:49 AM
- Karma Re: Lookup csv missing in definitions for liammcmenamin. 06-05-2020 12:48 AM
- Posted Re: DBConnect: OSError: [Errno 2] No such file or directory validate java command: /usr/java/latest/bin/java. on All Apps and Add-ons. 01-23-2020 01:24 PM
- Posted Re: How to upgrade multi-site indexer cluster WITH search head cluster, Splunk Enterprise 7.2 -> 7.3 on Installation. 12-17-2019 10:37 AM
- Posted How to upgrade multi-site indexer cluster WITH search head cluster, Splunk Enterprise 7.2 -> 7.3 on Installation. 12-16-2019 11:02 AM
- Tagged How to upgrade multi-site indexer cluster WITH search head cluster, Splunk Enterprise 7.2 -> 7.3 on Installation. 12-16-2019 11:02 AM
- Tagged How to upgrade multi-site indexer cluster WITH search head cluster, Splunk Enterprise 7.2 -> 7.3 on Installation. 12-16-2019 11:02 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 3 |
12-19-2023
12:09 PM
I opened a case with Splunk and they reviewed and replied that all of the DB Inputs running on the SHC Captain is expected behavior. Here's an excerpt from their findings: "In review and consultation with other colleagues I believe I may have found an answer. It is located in the documentation: https://docs.splunk.com/Documentation/DBX/3.15.0/DeployDBX/Distributeddeployment#:~:text=by%20executing%20input/output%20on%20the%20captain Specifically, the Deploy DB Connect on search head clusters section. DB Connect provides high availability on Splunk Enterprise with a Search Head Cluster, by executing input/output on the captain. Essentially, this is saying that in Splunk this is normal "expected behavior" and can be treated as such." To me this means that DB Inputs and DB Outputs on the Search Head Cluster will be limited by the hardware (CPU / Memory) of the captain, so you have to be careful with this. The benefit of DB Connect on an SHC is the replication of identity and connection configs across the cluster members. Rather than using DB Connect to configure and run the Input/output jobs, I recommend creating scheduled searches that run the dbxquery command. This way, the query jobs are distributed by the captain to all of the members of the cluster. I am testing this on my SHC with positive results!
... View more
12-14-2023
08:26 AM
Thanks, @gcusello , it looks like this might be the intended behavior, but seems odd that the captain would run all dbx jobs (unless the jobs are being delegated but the captain is doing all of the logging?) so I think I will open a case to get confirmation on expected behavior. We have always used dedicated Heavy Forwarders, but I figured it would be nice to maintain all identities, connections and inputs in one place! We'll see what Splunk says and I'll keep digging!
... View more
12-11-2023
04:02 PM
Hi all, I built a dedicated Search Head Cluster with 3 members and a deployer to load and test how DB Connect works in a shcluster. Splunk Enterprise 9.1.2 and DB Connect 3.15.1. The configs replicate fine across the members and I am running several inputs. It appears that all of the inputs so far are running on the captain only. I am wondering if this is normal behavior, and if the captain will start distributing input jobs to other members once it is maxed out? I am running this search to see the input jobs: index=_internal sourcetype=dbx_job_metrics connection=* host IN (abclx1001,abclx1002,abclx1003) | table _time host connection input_name db_read_time status start_time end_time duration read_count write_count error_count | sort - _time All inputs are successful, and the host field is always the same - it is the captain. The other members give me messages like this: 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 74 : Run DB Input name=test_db_input took 0.045278310775756836 s 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 66 : Input was run on other node status=303 content=b'Ignoring input request as other node is the captain' 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 51 : Run DB Input name=test_db_input 127.0.0.1 - - [11/Dec/2023:23:40:00 +0000] "POST /api/inputs/test_db_input/run HTTP/1.1" 303 51 "-" "python-requests/2.25.0" 41 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 45 : action=send_run_input_request 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 74 : Run DB Input name=test_db_input took 0.04212641716003418 s 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 66 : Input was run on other node status=303 content=b'Ignoring input request as other node is the captain' 127.0.0.1 - - [11/Dec/2023:23:40:00 +0000] "POST /api/inputs/test_db_input/run HTTP/1.1" 303 51 "-" "python-requests/2.25.0" 38 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 51 : Run DB Input name=test_db_input 2023-12-11T17:40:00-0600 [INFO] [dbx_db_input.py], line 45 : action=send_run_input_request Thoughts? Is the shc supposed to distributed these inputs the way it would distribute scheduled searches?
... View more
Labels
- Labels:
-
search head clustering
08-07-2023
08:21 AM
1 Karma
Yes - clearing my browser cache worked for me on upgrade from 3.2 to 3.13.
... View more
01-23-2020
01:24 PM
This worked for me!! I updated the script in 3.2 as you suggested.
... View more
12-17-2019
10:37 AM
I think I found it. There is a link I missed in the docs for "Perform a rolling upgrade of an indexer cluster".
This document, combined with the links in the OP will work for me I think.
Run Preliminary health checks
Upgrade the cluster master
Perform a rolling upgrade of a search head cluster
Perform a rolling upgrade of an indexer cluster
I might be able to get away with bringing down the indexers one site at a time, but not sure. Instead i'll probably go one-by-one before finalizing.
... View more
12-16-2019
11:02 AM
Upgrading from 7.2.5 to 7.3.3 to mitigate the Datetime.xml problem before the new year. I have a multi-site indexer cluster, five peers in site1, five peers in site2. I have a search head cluster, 6 members in site1 and 4 members in site2.
If I can use the site-by-site upgrade option, then I can keep ingesting data and maintain integrity, I never have to bring down all indexers at once. However, this option doesn't seem to account for a search head cluster, where there is also a deployer to worry about. It seems the other option is to upgrade in tiers. This option accounts for the deployer and I can do a rolling restart of the search head members, but the indexers must be brought down all at once. Am I missing something in the docs? Or is it acceptable to somehow combine the two by nesting the site-by-site indexer upgrade within the tiered upgrade? Like this: 1. Upgrade the Cluster Master 2. Perform a rolling upgrade of the search head cluster
a. Upgrade a non-captain member b. Upgrade the other members c. Upgrade the deployer d. Finalize the rolling upgrade 3. Upgrade site1 indexers 4. Upgrade site2 indexers Thanks for any help!
... View more
Labels
- Labels:
-
indexer
-
search head
-
upgrade
10-20-2019
12:04 PM
Hi @iamsplunker31, you're right, my answer was lacking. I made some edits to account for the different time ranges needed throughout the search. Does this help?
... View more
10-18-2019
09:38 PM
Hi @iamsplunker31, you'll want to add a time picker for the user to be able to choose the day. Then you can explicitly reference the time picker using the tokens, but add relative time modifiers after each token as needed, like this.
<form>
<label>Test Search</label>
<fieldset submitButton="false">
<input type="time" token="time">
<label></label>
<default>
<earliest>-d@d</earliest>
<latest>@d</latest>
</default>
</input>
</fieldset>
<row>
<panel>
<table>
<search>
<query>index=foo sourcetype=xyz | bin _time span=5m | stats dc(ecn) as Current by _time
| appendcols [search index=foo sourcetype=xyz earliest=$time.earliest$-7d@d latest=$time.latest$-7d@d | bin _time span=5m | stats dc(ecn) as LastWeek by _time]
| appendcols [search index=foo sourcetype=xyz earliest=$time.earliest$-14d@d latest=$time.latest$-14d@d | bin _time span=5m | stats dc(ecn) as TwoWeeksAgo by _time]
| appendcols [search index=foo sourcetype=xyz earliest=$time.earliest$-21d@d latest=$time.latest$-21d@d | bin _time span=5m | stats dc(ecn) as ThreeWeeksAgo by _time]
| appendcols [search index=foo sourcetype=xyz earliest=$time.earliest$-28d@d latest=$time.latest$-28d@d | bin _time span=5m | stats dc(ecn) as FourWeeksAgo by _time]
| eval AvgOfFourWeeks=(LastWeek+TwoWeeksAgo+ThreeWeeksAgo+FourWeeksAgo)/4
| eval Diff1=LastWeek-Current | eval Diff2=AvgOfFourWeeks-Current
| eval Est_Impact= Diff1-Diff2</query>
<earliest>$time.earliest$</earliest>
<latest>$time.latest$</latest>
</search>
<option name="refresh.display">progressbar</option>
</table>
</panel>
</row>
</form>
This is the simplest option. You could improve on the efficiency of this search by combining the appendcols into a single base search. However, you'd have to use streamstats or some other way to differentiate between the weeks so you can do math between them. I'll think on this one some more.
... View more
10-18-2019
09:20 PM
Hi @kishan2356, this code worked for me when I pasted it into an empty dashboard. It is assumed that the user enters relative time modifiers for earliest and latest times. Additional code would have to be added to account for different ways the user might use the time input, such as epoch time values, use of "now", etc. This uses a subsearch to pull last weeks data. So the search uses the time input values and the subsearch uses modified time input values.
<form>
<label></label>
<fieldset submitButton="false">
<input type="time" token="field1">
<label></label>
<default>
<earliest>-24h@h</earliest>
<latest>now</latest>
</default>
</input>
</fieldset>
<row>
<panel>
<table>
<search>
<query>index=some_index | stats count | addinfo | eval info_min_time_string=strftime(info_min_time,"%Y-%m-%d %H:%M:%S") | eval info_max_time_string=strftime(info_max_time,"%Y-%m-%d %H:%M:%S") | eval time_input_min_time_string=strftime(relative_time(now(),$field1.earliest|s$),"%Y-%m-%d %H:%M:%S") | fields time_input_min_time_string info_min_time_string info_max_time_string count
| append [search index=some_index earliest=$field1.earliest$-7d latest=$field1.latest$-7d | stats count | addinfo | eval info_min_time_string=strftime(info_min_time,"%Y-%m-%d %H:%M:%S") | eval info_max_time_string=strftime(info_max_time,"%Y-%m-%d %H:%M:%S") | eval time_input_min_time_string=strftime(relative_time(now(),$field1.earliest|s$),"%Y-%m-%d %H:%M:%S") | fields time_input_min_time_string info_min_time_string info_max_time_string count]</query>
<earliest>$field1.earliest$</earliest>
<latest>$field1.latest$</latest>
<sampleRatio>1</sampleRatio>
</search>
<option name="drilldown">none</option>
<option name="refresh.display">progressbar</option>
</table>
</panel>
</row>
</form>
... View more
10-18-2019
08:32 PM
2 Karma
Hi @danielbb, from the CLI, go to Settings > Server settings > General settings > Search preferences. There's a drop down there to change the default time range. The documentation to do this via web, CLI or conf file is here: https://docs.splunk.com/Documentation/Splunk/latest/Admin/Changedefaultvalues#Set_the_default_time_range
... View more
06-20-2019
10:06 AM
Adding an answer, although this post is old...
This appeared to be an issue between Splunk and Chrome. Using Chrome debugger, I could see that Splunk may have been attempting to open more than 6 connections (sockets??), and Chrome only allows 6. This error did not occur in IE or Firefox, which both allow more connections, like ~13.
We updated Splunk to the latest version at the time, which I think was 7.1. The problem persisted. I made sure to use Firefox after this, when making edits in DB Connect. Eventually, I tried again in Chrome and there is no issue. Not sure if an update to Chrome fixed this, or an update to DB Connect, or something else.
... View more
08-17-2018
10:05 AM
I am trying to install the Hive driver so that I can send Splunk data to the company's hadoop instance.
I found two articles that had some good details. Each uses a different method. I tried both and I haven't yet been successful.
1) https://answers.splunk.com/answers/575488/db-connect-3-hive-connection.html
The answer in this thread seems to indicate that I can drop the connection jar file, HiveJDBC4.jar into $SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers/ and then the rest of the supporting files from the Cloudera zip file go into a new directory $SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers/HiveJDBC4-lib/.
I tried this and then reloaded the drivers page in Splunk web, Hive driver still shows not installed.
2) https://www.splunk.com/blog/2015/02/25/splunk-db-connect-cloudera-hive-jdbc-connector.html
This blog post indicated that the driver should go in $SPLUNK_HOME/etc/apps/dbx/bin/lib. I'm assuming that would mean $SPLUNK_HOME/etc/apps/splunk_app_db_connect/bin/lib. I don't have this path. I tried adding a new lib directory into the bin directory and then unzipping the Cloudera file here. THen I created database_types.conf and added it to $SPLUNK_HOME/etc/apps/splunk_app_db_connect/local/. I reloaded the drivers page and still not showing that hive is installed.
Any thoughts on what else I could try or what I am doing wrong?
Thanks!
... View more
05-31-2018
01:55 PM
3 Karma
I just upgraded two heavy forwarders from DB Connect 3.0.0 to 3.1.3. On both forwarders, I can create a new input and save it. But if I try to clone an input and save, or edit an existing input and save, the "Finish", the screen hangs for 30-60 seconds before an orange banner shows the error "Failed to fetch". The input never saves. Does not appear to be driver-specific.
I am able to use the CLI to edit db_inputs.conf.
I don't see any errors in the internal logs, not sure if boosting the logging level will show an error? Has anyone seen this?
To be fair, I had a similar problem saving before, while using DB Connect 3.0.0, but only about half of the time. In 3.0, there was no error visible, except when I click back to the query screen I would see good query results, but an "undefined" error was displayed below the results...
... View more
