Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About brent_weaver
brent_weaver
Builder
Member since:
04-28-2015
02-17-2021
Community Statistics
| Posts | 336 |
| Solutions | 5 |
| Karma Given | 1 |
| Karma Received | 59 |
| Member Since | 04-28-2015 |
Activity Feed
- Got Karma for How can I add HTML to the search page?. 06-04-2024 06:17 AM
- Got Karma for Re: How to bulk delete multiple hosts from Splunk that have not phoned home within a set interval?. 02-26-2024 11:32 AM
- Got Karma for How are people managing dynamic entities in iTSI?. 10-09-2021 12:14 PM
- Got Karma for How do I secure the event collector port 8088 with an ssl certificate?. 04-19-2021 04:10 PM
- Got Karma for How to edit inputs.conf to prevent WinNetMon from using up all my license?. 03-04-2021 05:32 AM
- Posted HEC HWF behind AWS Loadbalancer. ELB or ALB? on Deployment Architecture. 12-09-2020 04:58 PM
- Posted Re: Splunk UF Docker Image root process writing splunkd's stderr.log to stdout on Deployment Architecture. 12-04-2020 03:58 AM
- Posted Splunk UF Docker Image root process writing splunkd's stderr.log to stdout on Deployment Architecture. 12-03-2020 05:27 PM
- Posted Re: Unable to rewrite host meta key at ingestion on Getting Data In. 11-29-2020 09:04 AM
- Posted Re: Unable to rewrite host meta key at ingestion on Getting Data In. 11-29-2020 08:51 AM
- Posted Unable to rewrite host meta key at ingestion on Getting Data In. 11-29-2020 05:29 AM
- Posted Re: TIME_FORMAT in props.conf seems to not be working on Getting Data In. 11-27-2020 10:24 AM
- Got Karma for Re: TIME_FORMAT in props.conf seems to not be working. 11-26-2020 12:40 PM
- Got Karma for Re: TIME_FORMAT in props.conf seems to not be working. 11-26-2020 12:39 PM
- Posted Re: TIME_FORMAT in props.conf seems to not be working on Getting Data In. 11-26-2020 12:01 PM
- Posted Re: TIME_FORMAT in props.conf seems to not be working on Getting Data In. 11-26-2020 09:12 AM
- Posted Re: TIME_FORMAT in props.conf seems to not be working on Getting Data In. 11-26-2020 09:06 AM
- Got Karma for TIME_FORMAT in props.conf seems to not be working. 11-25-2020 04:13 PM
- Posted TIME_FORMAT in props.conf seems to not be working on Getting Data In. 11-25-2020 01:16 PM
- Posted Re: Need help writing input stanza for maillog on Getting Data In. 11-15-2020 03:01 PM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
10-03-2016
10:25 AM
My search head cluster is no longer working after an update from 6.4 to 6.5.0 (I think it was the update!). It seemed to work just fine after the update but then I get in today and it is not working. Here is the log messages:
10-03-2016 17:10:37.586 +0000 WARN DistributedPeerManagerHeartbeat - Send failure while pushing PK to search peer = http://10.0.8.7:8089 , Connect Timeout
10-03-2016 17:10:37.586 +0000 ERROR DistributedPeerManagerHeartbeat - Status 502 while sending public key to cluster search peer http://10.0.8.8:8089:
10-03-2016 17:10:37.586 +0000 WARN DistributedPeerManagerHeartbeat - Send failure while pushing PK to search peer = http://10.0.8.71:8089 , Connect Timeout
10-03-2016 17:10:37.586 +0000 ERROR DistributedPeerManagerHeartbeat - Status 502 while sending public key to cluster search peer http://10.0.8.7:8089:
Please advise, it seems as though something happened to SSL in the update.
... View more
10-02-2016
06:35 PM
It seems that it is best to create fields at search time as opposed to index time.!?!? I need to make a field named src be copied/renamed to source_ip. We need to do this to simplify our searches and I am sure it is not hard to do.
Thanks!
... View more
- Tags:
- splunk-enterprise
09-27-2016
08:55 AM
I have a nice CEF file that parses quite nicely except the date is burred deep in the file and is in epoch milliseconds. The field name is start, how do I make that my datetime stamp when this indexes.
Thanks!
... View more
- Tags:
- splunk-enterprise
09-21-2016
10:36 AM
I need to move index data from one environment to another while [hopefully] consolidating them into fewer indexes. Is there a way to extract the data and then in turn import it into whatever index I want? Will the metadata case as issue? What are my options?
... View more
Labels
- Labels:
-
indexer
09-21-2016
10:20 AM
1 Karma
I used this procedure to resolve this issue. It seems that somehow the input got checkpointed into 2021?!?!
Remove checkpoints
You can remove checkpoints by running the Splunk clean utility.
Caution: Be careful when removing checkpoints. Running the clean command removes your indexed data. For example, clean all removes ALL your indexed data.
For example, to remove checkpoints for a specific scheme:
splunk clean inputdata [<scheme>]
So to resolve this I ran:
splunk clean inputdata AzureTableStorage
This removed the check point file and splunk was in turn able to reset the checkpoint and carry on.
Thank you!
... View more
09-19-2016
01:45 PM
I think this has been resolved by clearing the checkpoint file. I am doing more testing but it does seem to be getting logs. I will conclude this post with what I did to fix it if it does in fact work.
THANK YOU!
... View more
09-19-2016
01:10 PM
THANK YOU so much for the great information!
I have already verified that the datetime in azure is correct.
No date in inputs.conf
Where is the checkpoint file? I cannot seem to find it even after having looked in the python script. Can this value be reset?
... View more
09-19-2016
09:50 AM
The issue seems to be that the date ranges are being picked as 2021-xx-xx?!?
If I look at the log file I see this:
09-16-2016 14:11:56.741 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,741 DEBUG AzureStorageTable:299 - Filter string: ***TIMESTAMP gt datetime'2028-06-17***T20:31:07.379676+00:00' and TIMESTAMP lt datetime'2028-06-20T08:31:07.379676+00:00'
Why is this happening? ENabled debug in the python script:
/opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py
Any advice is MUCH appreciated!
... View more
09-16-2016
07:37 AM
More information:
Here is from the log file:
09-16-2016 14:11:56.653 +0000 DEBUG ExecProcessor - ExecProcessorSharedState::addToRunQueue() path='python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py' restartTimerIfNeeded=1
09-16-2016 14:11:56.653 +0000 DEBUG ExecProcessor - adding "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" to runqueue
09-16-2016 14:11:56.653 +0000 DEBUG ExecProcessor - cmd='python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py' Added to run queue
09-16-2016 14:11:56.653 +0000 DEBUG ExecProcessor - Running: python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py on PipelineSet 0
09-16-2016 14:11:56.653 +0000 DEBUG ExecProcessor - PipelineSet 0: Created new ExecedCommandPipe for "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py", uniqueId=1283
09-16-2016 14:11:56.738 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,737 DEBUG AzureStorageTable:237 - Starting AzureStorageTable.py 344
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,738 DEBUG AzureStorageTable:369 - XML: found configuration
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:374 - XML: found stanza AzureStorageTable://LinuxsyslogVer2v0
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'access_key'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'access_key' -> 'WYIl4RC07eU+XYiie/I8pdXFxBkcwxlPgNdVw8EEGF92Di808BCk2i/rBiaEG4ygbywJzEPSisSQsg1fg5dyUQ=='
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'dateTimeColumn'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'dateTimeColumn' -> 'TIMESTAMP'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'host'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'host' -> 'hdopeussadiag1a'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'index'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'index' -> 'bitbucket'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'interval'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'interval' -> '60'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'pollingMinutes'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'pollingMinutes' -> '3600'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'sourcetype'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'sourcetype' -> 'azure:storage:table'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:380 - XML: found param 'storage_account'
09-16-2016 14:11:56.739 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,739 DEBUG AzureStorageTable:384 - XML: 'storage_account' -> 'hdopeussadiag1a'
09-16-2016 14:11:56.740 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,740 DEBUG AzureStorageTable:380 - XML: found param 'table_name'
09-16-2016 14:11:56.740 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,740 DEBUG AzureStorageTable:384 - XML: 'table_name' -> 'LinuxsyslogVer2v0'
09-16-2016 14:11:56.740 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,740 DEBUG AzureStorageTable:264 - dateTimeStart = '2016-09-14T14:11:56.740547'
09-16-2016 14:11:56.740 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,740 DEBUG AzureStorageTable:271 - No dateTimeStart in inputs.conf...
09-16-2016 14:11:56.741 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,741 DEBUG AzureStorageTable:290 - Marker found for table LinuxsyslogVer2v0: 2028-06-17T20:31:07.379676+00:00
09-16-2016 14:11:56.741 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,741 DEBUG AzureStorageTable:299 - Filter string: TIMESTAMP gt datetime'2028-06-17T20:31:07.379676+00:00' and TIMESTAMP lt datetime'2028-06-20T08:31:07.379676+00:00'
09-16-2016 14:11:56.744 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:11:56,744 INFO connectionpool:657 - Starting new HTTPS connection (1): hdopeussadiag1a.table.core.windows.net
09-16-2016 14:11:59.633 +0000 DEBUG ExecProcessor - cmd='python /opt/splunk/etc/apps/splunk_app_db_connect/bin/rpcstart.py' Not added to run queue
09-16-2016 14:12:00.277 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:12:00,277 DEBUG connectionpool:350 - "GET /LinuxsyslogVer2v0()?$filter=TIMESTAMP%20gt%20datetime%272028-06-17T20%3A31%3A07.379676%2B00%3A00%27%20and%20TIMESTAMP%20lt%20datetime%272028-06-20T08%3A31%3A07.379676%2B00%3A00%27 HTTP/1.1" 200 None
09-16-2016 14:12:00.278 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:12:00,278 INFO AzureStorageTable:303 - Query returned 0 results.
09-16-2016 14:12:00.278 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:12:00,278 DEBUG AzureStorageTable:336 - No results found. Checkpoiting the end date/time used for query: 2028-06-20T08:31:07.379676+00:00
09-16-2016 14:12:00.278 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py" 2016-09-16 14:12:00,278 DEBUG AzureStorageTable:343 - Ending AzureStorageTable.py 344
09-16-2016 14:12:00.290 +0000 DEBUG ExecProcessor - PipelineSet 0: Got EOF from "python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py", uniqueId=1283
09-16-2016 14:12:00.300 +0000 DEBUG ExecProcessor - PipelineSet 0: Ran script: python /opt/splunk/etc/apps/TA-Azure/bin/AzureStorageTable.py, took 3.647290 seconds to run, 0 bytes read
It seems that the date ranges are wrong!?!?
... View more
09-14-2016
09:41 AM
3 Karma
The world would be a better place if everyone was like the splunk community. THANK YOU!
... View more
09-14-2016
09:26 AM
Yes you are correct. I apologize for the lack of clarity.
... View more
09-14-2016
08:47 AM
I am having marginal success. We are writing out Linux VMs syslogs to table storage, which I can see with Azure Storage explorer but does not show up in Splunk after having added this table to storage account inputs. For storage account, do I put in the full URL or just the hostname?
Does anyone have experience with this?
... View more
09-12-2016
05:15 AM
Thank you all for the help... What you guys are saying makes total sense.
... View more
09-09-2016
07:46 AM
Timestamp worked perfectly, thank you!
... View more
09-09-2016
05:43 AM
I am trying to setup Azure Table storage logs. There is a field I do not know how to populate:
Date/Time Column *
A date/time column to use for checkpointing and quering for new data.
Does anyone have any idea what I put there? I cannot find anything in documentation about it.
Thanks!
... View more
09-09-2016
05:31 AM
Thank you all for taking the time to help me out here... thanks!
... View more
09-08-2016
10:01 AM
When setting up a Heavy forwarder, do I need to have the index created locally as I do in my indexer cluster? So I am setting up Splunk DB Connect and McAfee and have configured the Splunk server to be a HWP. I am testing writing to an index called bitbucket. In order for this to work, do I need to have a local index called bitbucket as I do in my indexer cluster? I have configured it to not keep a local copy.
Thanks!
... View more
09-07-2016
07:21 AM
I have an index cluster and am setting up volume management. I have modified SPLUNK_DB env variable to point to /data and also have volume defined in my index config pointing to the same location.
From /opt/splunk/etc/splunk-launch.conf:
# Version 6.4.3
# Modify the following line to suit the location of your Splunk install.
# If unset, Splunk will use the parent of the directory containing the splunk
# CLI executable.
#
# SPLUNK_HOME=/home/build/build-home/galaxy
# By default, Splunk stores its indexes under SPLUNK_HOME in the
# var/lib/splunk subdirectory. This can be overridden
# here:
#
# SPLUNK_DB=/home/build/build-home/galaxy/var/lib/splunk
# Splunkd daemon name
SPLUNK_SERVER_NAME=Splunkd
# Splunkweb daemon name
SPLUNK_WEB_NAME=splunkweb
# If SPLUNK_OS_USER is set, then Splunk service will only start
# if the 'splunk [re]start [splunkd]' command is invoked by a user who
# is, or can effectively become via setuid(2), $SPLUNK_OS_USER.
# (This setting can be specified as username or as UID.)
#
# SPLUNK_OS_USER
SPLUNK_OS_USER=splunksvc
SPLUNK_DB=/data
On the master index server, from the file /opt/splunk/etc/master-apps/VolMgtConfig/local/indexes.con:
## The following stanza defines the location of the indexes as well as manages the space
## This tells Splunk that the index location can only get to be 2.75 TB (2883584 Mb)
[volume:primary]
path = /data
maxVolumeDataSizeMB = 2883584
I am getting the following message when I restart the indexers (which makes sense):
09-06-2016 23:32:28.208 +0000 ERROR IndexConfig - idx=history Path coldPath='/data/historydb/colddb' (realpath '/data/historydb/colddb') is inside volume=primary (path='/data', realpath='/data'), but does not reference that volume. Spac
e used by coldPath will *not* be volume-mananged. Config error?
09-06-2016 23:32:28.208 +0000 ERROR IndexConfig - idx=main Path homePath='/data/defaultdb/db' (realpath '/data/defaultdb/db') is inside volume=primary (path='/data', realpath='/data'), but does not reference that volume. Space used by h
omePath will *not* be volume-mananged. Config error?
Understanding why this is doing this, do I care? Other than the fact that my _cluster indexes will not be managed by volume management, are there implications? If so what would be the best config to use to remedy this issue?
Thanks!
... View more
- Tags:
- splunk-enterprise
09-06-2016
02:27 PM
1 Karma
I am installing Splunk Enterprise Security 4.1.1 and know that this application can gobble up file system space. I have Splunk installed in /opt/splunk (which is typical) and need to know where ES keeps all these files. My idea is to add additional storage to the system and make a symlink to it so there is plenty of space for ES and not killing my root fs.
Any thoughts are welcomed!
... View more
09-06-2016
09:46 AM
We are trying to use the Splunk license app and it is not working. Is it possible that Splunk does not count its internal logs against the license?
... View more
09-02-2016
09:05 AM
We are in a slight dilemma where we are trying to reduce down the number of indexes we have, understanding that this impacts retention and security. Security is not really an issue, it is retention we are trying to tackle. Since everything is sourcetyped, is it possible to set retention periods on different sourcetypes? I am pretty sure I can write a search to to delete the data, but is there a better way?
... View more
08-31-2016
05:02 AM
There does seems to be a header:
-----BEGIN CERTIFICATE-----
And there is no ^M (CR) in the file.... This cannot be that difficult!
... View more
08-30-2016
12:15 PM
Now I am seeing this on the FWD servers from splunkd.log
08-30-2016 18:03:19.804 +0000 ERROR SSLCommon - Can't read key file /opt/splunk/etc/auth/fwd/star_gehccloud_com_public.pem errno=151441516 error:0906D06C:PEM routines:PEM_read_bio:no start line.
... View more
08-30-2016
10:54 AM
Hello.
We have multiple "customers" logging into our Splunk environment and we need to be able to track their license and storage consumption. There must be a way built into Splunk that can allow us to do this, we are in the process of setup, so the time is now 🙂
... View more
Labels
- Labels:
-
license
08-30-2016
05:04 AM
I need to be able to have port 8000 listen for just http and 8443 for HTTPS. How do I accomplish this?
... View more
- Tags:
- splunk-enterprise
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
02-17-2021
11:03 PM
|
