Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About gbowden_pheaa
gbowden_pheaa
Path Finder
Member since:
12-11-2012
06-05-2020
Community Statistics
| Posts | 24 |
| Solutions | 2 |
| Karma Given | 1 |
| Karma Received | 9 |
| Member Since | 12-11-2012 |
Activity Feed
- Got Karma for How to limit date timestamp parsing view?. 06-05-2020 12:47 AM
- Got Karma for SOS - How Do I Configure for a Distributed Environment?. 06-05-2020 12:47 AM
- Got Karma for Re: SOS - How Do I Configure for a Distributed Environment?. 06-05-2020 12:47 AM
- Got Karma for Re: SOS - How Do I Configure for a Distributed Environment?. 06-05-2020 12:47 AM
- Got Karma for How to change the default of 10 lines per page in forwarder management?. 06-05-2020 12:47 AM
- Got Karma for How to change the default of 10 lines per page in forwarder management?. 06-05-2020 12:47 AM
- Got Karma for How to change the default of 10 lines per page in forwarder management?. 06-05-2020 12:47 AM
- Got Karma for How to change the default of 10 lines per page in forwarder management?. 06-05-2020 12:47 AM
- Got Karma for How to change the default of 10 lines per page in forwarder management?. 06-05-2020 12:47 AM
- Karma Re: Blank Forwarder Management page on Splunk 6 for rabitoblanco. 06-05-2020 12:46 AM
- Posted Re: Can inputs.conf be reloaded without restarting splunkd? on Getting Data In. 06-11-2018 01:06 PM
- Posted Re: Issue in Data Parsing for extracting the field . on Archive. 06-05-2018 01:28 PM
- Posted Re: Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 12:36 PM
- Posted Re: Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 12:34 PM
- Posted Re: Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 11:27 AM
- Posted Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 08:43 AM
- Tagged Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 08:43 AM
- Tagged Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 08:43 AM
- Tagged Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 08:43 AM
- Tagged Configuring Splunk for Multiple Indexer Partitions on Getting Data In. 05-21-2018 08:43 AM
06-11-2018
01:06 PM
For many cases the extracts in the props.conf can be reloaded by appending the following to a search -
| extract reload=true
This may work for transforms, if there is only a change to an existing setting, You would probably need to restart if you create a new setting in transforms.conf . Then again, it might not. I have had mixed results with many of these "on the fly" efforts. It is safest to restart.
... View more
06-05-2018
01:28 PM
Generally speaking, you need to define the event boundaries (event beginning and end) and date/time configurations in your props.conf file. Since this in not a single line event, you may need to use the more expensive (in processing) line break definitions such as "BREAKONLYBEFORE" or "MUST BREAKAFTER" and the regex required to identify the event boundaries. You also need to define the date/time location in your data (TIMEPREFIX and/or TIMESTAMPLOOKAHEAD) , and then the date/time format (TIMEFORMAT). You may also need to modify the TRUNCATE or MAXEVENTS defaults if the event can be large (10000 bytes for TRUNCATE and 256 lines for MAX_EVENTS).
Going through the Admin guide in the props.conf specifications should help you along this process.
http://docs.splunk.com/Documentation/Splunk/7.1.1/Admin/Propsconf
This is just a starting point, not much more can be done without a data sample.
... View more
05-21-2018
12:36 PM
solarboyz1 provided the answer - I was not referencing the correct path in my cold path, so the data was not being seen. Thank you!
... View more
05-21-2018
12:34 PM
You are right on - I tried both changing the colddb name from index/db to index/colddb, and changing the indexes.conf to use coldpath = volume:cold/index/db, and both worked. Thank you!
Now the big question, we do not freeze data, we roll it off. Will there be advantage to one method over the other in data retention and retention processing? I need to dig more into this to determine of data can roll from volume:hotwarm/index/db to volume:cold/index/db to see if there are issues, or if the data has to roll from the colddb directory.
Thank you, this is a huge help!
... View more
05-21-2018
11:27 AM
We have a non-root user that owns the /splunkdata partition, with 700 rights. We are not seeing errors regarding the cold location. The cold location was the previous primary partition of all indexes, we've added a "flash" drive to handle the hot/warm buckets. Hot/Warm appears to be running fine. There are numerous buckets in the various indexes in the cold partition.
... View more
05-21-2018
08:43 AM
I am not sure how to configure the indexes.conf AND the splunk-launch.conf. I understand multiple volumes in indexes.conf, such as:
[volume:hotwarm]
path = /splunkindexes/hot
[volume:cold]
path = /splunkdata
and using this in the index definition in indexes.conf:
[main]
homePath = volume:hotwarm/defaultdb/db
coldPath = volume:cold/defaultdb/colddb
thawedPath = $SPLUNK_DB/defaultdb/thaweddb
repFactor = auto
I have the splunk-launch.conf set as:
SPLUNKHOME=/splunk
SPLUNKDB = /splunkindexes/hot
setting the SPLUNK_DB to the hotwarm volume.
I am not seeing the cold volume data. I just made these changes, and have the hot to cold rollover set "disk size", so it will stay hot until the disk is close to full then it will roll to cold.
Any ideas on way I cannot see the existing cold data? Thanks...
... View more
03-23-2017
01:34 PM
We are trying to connect a Mobileiron built-in Splunk forwarder to an indexer cluster. At best we get an intermittent connection. Whenever we recycle the Mobileiron splunk daemon we see the following message from an indexer:
timestamp INFO TcpInputProc - clustering is enabled but ACK not enabled on forwarder=ip_address
We do not get any other messages after this.
Shouldn't an indexer accept a connection whether useAck is loaded or not from a forwarder? Are there any indexer configurations to accept useAck=false (default)?
... View more
03-29-2016
12:34 PM
My issue was unfortunately not listed in the configs provided. In my props.conf file I has specified a field delimiter, the data files did not have a header, so this configuration item broke the entire data collection. Once this was removed everything works as expected.
Some of the events used a "~" as a delimiter, I ended up using search time extractions to pull the fields.
Thank you for you help...
... View more
02-24-2016
07:41 AM
There are non "INDEXED_EXTRACTIONS". Using the
https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus
command I get the following:
e/notes/notesr4/IBMTECHNICALSUPPORT/console.log
file position 65536
file size 10485760
percent 0.62
type open file
I am not sure why this is stopping at exactly 64kb, but it does not pull any data. Here's a sample of the file:
console_kodiak_2016_02_14@05_33_10.log
[6684810:00009-01286] 02/21/2016 12:25:37 DIIOP Server: 10.109.6.122 connected
[6684810:00009-01286] 02/21/2016 12:25:37 DIIOP Server: 10.109.6.122 disconnected
[7667964:00121-29813] 02/21/2016 12:27:33 0 Transactions/Minute, 0 Notes Users
[7929956:00002-00001] 02/21/2016 12:27:44 AMgr: Start executing agent 'Process PSXActions Process PSXActions' in 'testarea/APSvNew.nsf' by Exec
utive '1'
[7929956:00002-00001] 02/21/2016 12:27:44 AMgr: 'server/domain' is the agent signer of agent 'Process PSXActions Process PSXActions' in 'testarea/
APSvNew.nsf'
[7929956:00002-00001] 02/21/2016 12:27:44 AMgr: 'Agent 'Process PSXActions Process PSXActions' in 'testarea/APSvNew.nsf' will run on behalf of
'kodiak/AES'
[7929956:00002-00001] 02/21/2016 12:27:44 AMgr: Agent 'Process PSXActions Process PSXActions' in database 'testarea/APSvNew.nsf' signed by 'kod
iak/AES' is running in Full Administrator mode
[7929956:00002-00001] Agent 'Process PSXActions|Process PSXActions' calling script library 'lsFormNewResourceRequest': Agent signer 'CN=server/O=domain', Script library signer 'CN=server/O=domain'
Thank you for helping, I getting much more from you than I am Splunk Support.
... View more
02-22-2016
11:19 AM
Does Notes lock the console.log file so that Splunk cannot read this file?
Further investigation into this has "changed" my understanding of the symptoms. The file in question is a running log of Lotus Notes core functions. It appends until this file reaches either a size or date limit, then archives the file and starts a new file. Here's what I know -
The console.log file will not forward data to the indexers, all other splunk files will from the same server. This tells me the base configurations on the forwarder are correct.
Running a "splunk show monitor: lists the file we want to monitor
Copying the active file to the /tmp directory still does not allow monitoring
Copying his copy to another splunk system (dev) does not allow splunk to monitor this file
Copying an archived file to the dev system DOES read and ingest the file
Permissions to allow Splunk to traverse the file system to this file are in place, but one of the directories is hidden - there is an extended ACL on this directory to allow traversing across it.
I am suspecting that Notes is appending this file in an unexpected method, such as it keeps the file open and does not actually append to this file.
Is Splunk expecting an "end of file" on the file to monitor it?
... View more
02-10-2016
09:47 AM
I have verified permissions are not an issue, the full path is navigable, and the data can be "cat'd" either through direct access or navigation to the file.
I am getting a new error -
02-10-2016 12:38:57.702 -0500 ERROR TailingProcessor - Ignoring path="/home/notes/notesr4/IBMTECHNICALSUPPORT/console.log" due to: Bug during applyPendingMetadata, header processor does not own the indexed extractions confs.
Not finding anything here in answers that seems to apply to his error. Thank you in advance...
... View more
02-08-2016
11:13 AM
I just tried copying the file to /tmp and it still does not ingest it. Officially stumped...
... View more
02-08-2016
10:08 AM
Permissions are find, and there are no symbolic links in the path. Copying the file out to a test Splunk system handles the file without issue. I can "cat" the file also without issue under the user account running splunk.
I have installed AIX forwarders on dozens of servers here, this is the first time I've had this issue.
Thank you dwaddle for your answer. any other ideas?
... View more
02-05-2016
10:02 AM
I have several AIX forwarders that are not liking what I think is a simple monitor. We are looking for 1 file to ingest, the read permissions are in place for the file. The _internal files are being ingested. From the forwarder everything appears correct, and from the deployment server everything appears correct. Only the 1 desired file is not being monitored. Please tell me what I am doing wrong.
Forwarder version 6.3.1
Here's the inputs.conf
[monitor:///home/notes/notesr4/IBM_TECHNICAL_SUPPORT/console.log]
_TCP_ROUTING=test_cluster
index=t_webapp
sourcetype=COTS:Notes
Here's the outputs.conf
[tcpout]
defaultGroup = test_cluster
indexAndForward = false
[tcpout:test_cluster]
server = indexer1:9997,indexer2:9997
autoLB = true
useACK = true
maxQueueSize = 7MB
Here are the _internal messages types:
02-05-2016 12:35:53.181 -0500 ERROR TailingProcessor - Ran out of data while looking for end of header
02-05-2016 09:43:00.622 -0500 WARN DistributedPeerManager - feature=DistSearch not enabled for your license level
02-05-2016 09:43:00.349 -0500 WARN ClusteringMgr - Ignoring clustering configuration, the active license disables this feature.
02-05-2016 09:42:57.293 -0500 WARN ulimit - Core file generation disabled
I'm looking into the ulimit message
Here are the forwarder CLI results for forward-server and monitor
kodiak:/splunkforwarder/bin> ./splunk list forward-server
Active forwards:
indexer1:9997
Configured but inactive forwards:
indexer2:9997
kodiak:/splunkforwarder/bin> ./splunk list monitor
Monitored Directories:
$SPLUNK_HOME/var/log/splunk/splunkd.log
/splunkforwarder/var/log/splunk/audit.log
/splunkforwarder/var/log/splunk/btool.log
/splunkforwarder/var/log/splunk/conf.log
/splunkforwarder/var/log/splunk/first_install.log
/splunkforwarder/var/log/splunk/license_audit.log
/splunkforwarder/var/log/splunk/license_usage.log
/splunkforwarder/var/log/splunk/metrics.log
/splunkforwarder/var/log/splunk/mongod.log
/splunkforwarder/var/log/splunk/remote_searches.log
/splunkforwarder/var/log/splunk/scheduler.log
/splunkforwarder/var/log/splunk/searchhistory.log
/splunkforwarder/var/log/splunk/splunkd-utility.log
/splunkforwarder/var/log/splunk/splunkd.log
/splunkforwarder/var/log/splunk/splunkd_access.log
/splunkforwarder/var/log/splunk/splunkd_stderr.log
/splunkforwarder/var/log/splunk/splunkd_stdout.log
/splunkforwarder/var/log/splunk/splunkd_ui_access.log
$SPLUNK_HOME/var/spool/splunk/...stash_new
Monitored Files:
$SPLUNK_HOME/etc/splunk.version
/home/notes/notesr4/IBM_TECHNICAL_SUPPORT/console.log
All suggestions are welcome...thanks in advance
... View more
11-23-2015
11:15 AM
The above methods are likely fine for smaller environments, but not very easy to address in larger or "overworked" clusters. I've deployed a new index file to the indexer WITHOUT the index to be cleaned (apply cluster-bundle), wait for the system to stablize, then use the OS to delete the index files (default /opt/splunk/var/lib/indexname). Once stabilized, deploy the original index file.
I have not had problems (like the delete command crashing the whole system) with this approach, it seems to be as fast and probably more efficient than the that techniques I've tried.
... View more
02-03-2015
08:00 AM
Now that I have moved to a search head cluster from a search head pooling (v6.1.1 to v6.2.1), I am getting multiple sent alerts for a single alert. I was able to control this in 6.1.1 by enabling only 1 search head to send e-mail, but would this approach work in a cluster?
I am confused because I have 3 search heads in the cluster, but the cluster sends 2 of each alert, not 1 or 3 as I would expect.
Is there a way to determine which search head actually sends the alerts?
martin_muellar, would you explain why you feel the configuration in a cluster is irrelevant? It was my understanding the SH cluster captain would manage this, but I obviously have a disconnect somewhere.
Also - how should app objects created by users, specifically alerts, be managed if differing configurations are used to control this situation?
Thanks to all in advance.
... View more
08-18-2014
07:20 AM
5 Karma
Is there a way to change the default of "10 lines" in Forwarder Management? I find it extremely annoying that this page, unlike most other pages in Splunk, will not remember the last setting for this area. I have many more forwarders, applications, and server classes than 10, and it really slows me down when I have to add many new apps/server classes, which presently is often.
... View more
08-11-2014
10:29 AM
2 Karma
I've made some headway -
I edited the /$SPLUNKAPP/etc/apps/sos/lookups/splunkservers_cache.csv file by adding the additional search heads into the list. SOS does not report the server hardware and OS statistics, but I can now see the search statistics from my master node.
That's mostly want I wanted.
... View more
08-11-2014
10:13 AM
1 Karma
I have multiple search heads and multiple indexers in a cluster. I've deployed SOS to the cluster master (also the deployment server and license manager), and to all search heads. I've deployed the TA-SOS to all indexers, enabled scripting on all servers, and restarted the entire environment.
All _internal data is being sent to the cluster.
On my master node, SOS shows only itself and the search peers. This is the same for the rest of the search heads.
I want to be able to see everything from my master node. Am I hoping for too much, or have a missed something in the configurations?
Thanks...
... View more
08-06-2014
05:34 AM
This got me going - thank you!
I was editing only the deployment-apps directory. I renamed the serverclass.conf file and restarted Splunk, that got me going again. I am rebuilding this file through the GUI, I read somewhere you shouldn't edit the serverclass.conf file for just this reason. Apparently, one must be careful will the deployment-apps directory also!
Thanks again!
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
06-05-2020
02:04 AM
|
