Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About sarnagar
sarnagar
Contributor
Member since:
03-20-2015
06-05-2020
Community Statistics
| Posts | 95 |
| Solutions | 1 |
| Karma Given | 36 |
| Karma Received | 20 |
| Member Since | 03-20-2015 |
Activity Feed
- Karma Re: What si "All non Internal Indexes" in splunk, that is present under Indexes in a Role for teunlaan. 06-05-2020 12:49 AM
- Karma Re: extract a string from source and table the list for the associated hosts. for peterchenadded. 06-05-2020 12:49 AM
- Karma Re: Can we limit the number of rows/events in a statistics table dashboard panel? for gcusello. 06-05-2020 12:49 AM
- Karma Re: Tabulate the list of folders under a directory for all hosts? for somesoni2. 06-05-2020 12:49 AM
- Karma Re: Tabulate the list of folders under a directory for all hosts? for niketn. 06-05-2020 12:49 AM
- Karma Re: splunk analytics for hadoop - kerberos for rdagan_splunk. 06-05-2020 12:49 AM
- Got Karma for Archive Splunk buckets to AWS S3?. 06-05-2020 12:49 AM
- Got Karma for Archive Splunk buckets to AWS S3?. 06-05-2020 12:49 AM
- Karma Re: Hunk: What does the vix.splunk.home.hdfs setting refer to? for rdagan_splunk. 06-05-2020 12:48 AM
- Karma Hunk: What does the vix.splunk.home.hdfs setting refer to? for Harishma. 06-05-2020 12:48 AM
- Karma Why do I receive "BlockMissingException - Could not obtain block" error when trying to search Hadoop virtual index? for sbrice. 06-05-2020 12:48 AM
- Karma Re: Why do I receive "BlockMissingException - Could not obtain block" error when trying to search Hadoop virtual index? for rdagan_splunk. 06-05-2020 12:48 AM
- Karma Re: How to exclude certain fields in json logs from being displayed on UI OR being indexed? for sundareshr. 06-05-2020 12:48 AM
- Karma Re: Splunk Add-on for Kafka: How to resolve multiple errors "No leader topic jsontest partition 0"? for rdagan_splunk. 06-05-2020 12:48 AM
- Karma Re: Can I generate the HTTP Event Collector token on the Heavy Forwarder or Search Head? for somesoni2. 06-05-2020 12:48 AM
- Karma Re: Splunk Add-on for Kafka: Does managing inputs manually from each Heavy Forwarder create duplicates? for rpille_splunk. 06-05-2020 12:48 AM
- Karma Re: DbConnect 2.2 to 2.4 - Unable to decrypt password for splunk_kk. 06-05-2020 12:48 AM
- Karma Splunk DB Connect 3: How to resolve "Error in 'dbxquery' command: Could not find file: 'java.path'" for bjmclean. 06-05-2020 12:48 AM
- Karma How to resolve Unable to initialize modular input "server" defined inside the app "splunk_app_db_connect" for teddyidc1101. 06-05-2020 12:48 AM
- Karma Re: Identify searches that take long time in a SH Cluster. for ctaf. 06-05-2020 12:48 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
08-02-2018
02:35 AM
1) Firstly
Whats the difference between
"All non Internal Indexes"
and
"All Internal Indexes"
2)
I have 3 Roles "A" and "B" and "C"
Role "A" - with
Capabilities-
accelerate_search
change_own_password
edit_search_schedule_window
export_results_is_visible
extra_x509_validation
get_metadata
get_typeahead
input_file
output_file
pattern_detect
request_remote_tok
rest_apps_view
rest_properties_get
rest_properties_set
schedule_rtsearch
search
Default index - main
the Indexes selected as "All non Internal Indexes"
Role B
inherit role: A
Capability:
schedule_search
rtsearch
Index Default: A
Indexes: A, B
Role C;
inherit role: A
Capability:
schedule_search
rtsearch
Indexes default: C
Indexes: B,C
A test user1 with role A can view index A
user2 with role B can view index A
But user3 with role C cant view index A.
Why? user3 inherits Role A
... View more
06-05-2018
06:51 AM
We have applications on prem as well on AWS.
Can I have single Splunk Enterprise (multi-site) set up both on - ON Prem as well as AWS?
For ex. Site 1 is ON PREM and Site-2 is AWS...
EC2 instances can forward to Indexers on AWS Site -2 and on-prem servers to site-1 Indexers?
Should my Search head cluster also be spread across on-prem and AWS?
Or can be purely be set on AWS alone vice-versa?
What contraints should I consider?
Kindly guide on suggestions of setting up a hybrid environment..
... View more
- Tags:
- aws
- splunk-enterprise
05-17-2018
04:08 AM
Search peer indexer1 has the following message: License Manager: Failed to contact license master: reason='Unable to connect to license master: https://xxx.com:8089 Connect to=https://xxx.com:8089 C timed out; exceeded 30sec', first failure time=1526522560 (Wed May 16 22:02:40 2018 EDT).
What does this banner msg indicate?
pass4SymmKey were set up between the license master and the Indexers...
What does this sudden banners msgs indicate and why do we get these all of sudden, which was fine till date.
Kindly help
... View more
03-06-2018
08:00 AM
I'm using clustered env.
I've placed props n transforms on heavy forwarder
... View more
03-06-2018
04:56 AM
Hi @493669 ,
It still doesnt work. It nulls entire log.
Here is the complete sample log
20180305 06:10:43,769 EST DEBUG sql.Connection [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {conn-11499536} Connection
20180305 06:10:43,769 EST DEBUG sql.Connection [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {conn-11499536} Preparing Call: { call SA_TAO_AIP_CASH_MGT_TXN_PKG.get_cash_txn_dtl_by_cp_id(?,?) }
20180305 06:10:43,769 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499537} Executing Statement: { call SA_TAO_AIP_CASH_MGT_TXN_PKG.get_cash_txn_dtl_by_cp_id(?,?) }
20180305 06:10:43,769 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499537} Parameters: [55858397233]
20180305 06:10:43,769 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499537} Types: [java.lang.Long]
20180305 06:10:43,774 EST DEBUG sql.Connection [WorkflowManager_10027887_55656942908_55666096052_3006] - {conn-11499538} Connection
20180305 06:10:43,774 EST DEBUG sql.Connection [WorkflowManager_10027887_55656942908_55666096052_3006] - {conn-11499538} Preparing Call: { call SA_TAO_AIP_COMMON_PKG.GET_PORTF_INSTANCE_LOTS(?, ?) }
20180305 06:10:43,774 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55656942908_55666096052_3006] - {pstm-11499539} Executing Statement: { call SA_TAO_AIP_COMMON_PKG.GET_PORTF_INSTANCE_LOTS(?, ?) }
20180305 06:10:43,774 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55656942908_55666096052_3006] - {pstm-11499539} Parameters: [398207947]
20180305 06:10:43,774 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55656942908_55666096052_3006] - {pstm-11499539} Types: [java.lang.Long]
20180305 06:10:43,791 EST INFO domain.CashDomain [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - Available Cash for [investablePortfolioId=55868440408, sleeve=ROP, lot=INITIAL, withLiq=false] is: 1219.5500000
20180305 06:10:43,791 EST DEBUG sql.Connection [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {conn-11499540} Connection
20180305 06:10:43,791 EST DEBUG sql.Connection [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {conn-11499540} Preparing Call: { ? = call sa_tao_aip_parameter_pkg.get_ip_portf_param_value(?, ?) }
20180305 06:10:43,791 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499541} Executing Statement: { ? = call sa_tao_aip_parameter_pkg.get_ip_portf_param_value(?, ?) }
20180305 06:10:43,791 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499541} Parameters: [CASH_TXN_THRESHOLD_DAYS, 55868440408]
20180305 06:10:43,791 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499541} Types: [java.lang.String, java.lang.Long]
20180305 06:10:43,801 EST INFO domain.CashDomain [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - Cash transaction threshold days value for [investablePortfolioId=55868440408] is: 60
20180305 06:10:43,802 EST DEBUG sql.Connection [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {conn-11499542} Connection
20180305 06:10:43,802 EST DEBUG sql.Connection [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {conn-11499542} Preparing Call: { call sa_tao_aip_portf_metric_pkg.ins_portf_metric_val(?) }
20180305 06:10:43,802 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499543} Executing Statement: { call sa_tao_aip_portf_metric_pkg.ins_portf_metric_val(?) }
20180305 06:10:43,802 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499543} Parameters: [oracle.sql.ARRAY@10b46d2d]
20180305 06:10:43,802 EST DEBUG sql.PreparedStatement [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - {pstm-11499543} Types: [oracle.sql.ARRAY]
20180305 06:10:43,829 EST INFO workflow.TransactionalWorkflowOperation [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - Returning to Workflow manager, returnCode[0]
20180305 06:10:43,829 EST INFO workflow.TransactionalWorkflowOperation [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180305 06:10:43,848 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499544} Connection
20180305 06:10:43,848 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499544} Preparing Call: { call SA_TAO_AIP_PORTFOLIO_PKG.GET_ALL_INV_PORTF_FOR_CLIENT(?, ?) }
20180305 06:10:43,848 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499545} Executing Statement: { call SA_TAO_AIP_PORTFOLIO_PKG.GET_ALL_INV_PORTF_FOR_CLIENT(?, ?) }
20180305 06:10:43,848 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499545} Parameters: [55858397233]
20180305 06:10:43,848 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499545} Types: [java.lang.Long]
20180305 06:10:43,856 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499546} Connection
20180305 06:10:43,856 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499546} Preparing Call: { call SA_TAO_AIP_OPT_TEMPLATE_PKG.get_portf_strtgy_ovrrde(?,?) }
20180305 06:10:43,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499547} Executing Statement: { call SA_TAO_AIP_OPT_TEMPLATE_PKG.get_portf_strtgy_ovrrde(?,?) }
20180305 06:10:43,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499547} Parameters: [55868440408]
20180305 06:10:43,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499547} Types: [java.lang.Long]
20180305 06:10:43,867 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499548} Connection
20180305 06:10:43,867 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499548} Preparing Call: { call sa_tao_aip_portf_metric_pkg.get_portf_metric_val(?, ?, ?) }
20180305 06:10:43,867 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499549} Executing Statement: { call sa_tao_aip_portf_metric_pkg.get_portf_metric_val(?, ?, ?) }
20180305 06:10:43,867 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499549} Parameters: [10027887, 55868440408]
20180305 06:10:43,867 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499549} Types: [java.lang.Long, java.lang.Long]
... View more
03-05-2018
05:25 AM
Hi @493669 ,
I tried removing the setparsing part and it removes all the events and nothing is indexed.
I need only lines that contain those phrases to be removed.
Need to ignore lines containing below words:
"Preparing Call"
"Clearing Module Context"
"Types: []"
"Committing all transactions using () transaction manager"
Regex used:
(Committing all transactions using (.) transaction manager)|(Types:\s[.])|(Preparing Call)|(Clearing Module Context)
Ex of Logfile:
20180302 05:02:28,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_55808724927] - {pstm-47402055} Types: [java.lang.Long, java.lang.String, java.lang.String, java.lang.Long, null, null, java.lang.String, java.lang.String, null, java.sql.Timestamp, null, null, java.lang.Long, null, null, null]
20180302 05:02:28,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_58055999998_58072708329_7001] - {pstm-47402058} Types: [java.lang.Long]
20180302 05:02:28,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_61129156140] - {pstm-47402059} Types: [java.lang.Long]
20180302 05:02:28,858 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_61129156140] - {pstm-47402061} Types: [java.lang.Long, java.lang.String, java.lang.String, java.lang.Long, null, null, java.lang.String, java.lang.String, null, java.sql.Timestamp, null, null, java.lang.Long, null, null, null]
20180302 05:02:28,859 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_58055999998_58072708329_7001] - {pstm-47402064} Types: [java.lang.Long]
20180302 05:02:28,859 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_55808724927] - {pstm-47402065} Types: [java.lang.Long, java.lang.Long, java.lang.Long]
20180302 05:02:28,866 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_61129156140] - {pstm-47402068} Types: [java.lang.Long, java.lang.Long, java.lang.Long]
20180302 05:02:28,866 EST DEBUG sql.PreparedStatement [ConstraintManagerService_10027886_59615007125_59266246402_3003] - {pstm-47402075} Types: [java.lang.Long, null, null]
20180302 05:02:28,866 EST DEBUG sql.PreparedStatement [ConstraintManagerService_10027886_35770283302_35772759397_3003] - {pstm-47402079} Types: [java.lang.Long, null, null]
20180302 05:02:28,866 EST DEBUG sql.PreparedStatement [WorkflowManager_10027886_58055999998_58072708329_7001] - {pstm-47402081} Types: [java.lang.Long]
20180302 05:02:28,866 EST DEBUG sql.PreparedStatement [ConstraintManagerService_10027886_45166366355_45172697064_3003] - {pstm-47402074} Types: [java.lang.Long, null, null]
For NOT "Committing all transactions using (*) transaction manager"
20180302 05:04:27,239 EST INFO workflow.TransactionalWorkflowOperation [InvestablePortfolioConstraints_10027886_56399425167_56411171958] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180302 05:04:27,253 EST INFO workflow.TransactionalWorkflowOperation [OptimizerService_10027886_63883247085_62514894824_4010] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180302 05:04:27,255 EST INFO workflow.TransactionalWorkflowOperation [ClientPortfolioDatamartLoadService_10027886_55546646750_55555552651_1999] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180302 05:04:27,266 EST INFO workflow.TransactionalWorkflowOperation
Clearing Module Context asdfhasduoifhuiase\djlkfgasdui [PostOptPortfolioScores_10027886_42388259809_42393917628_8008] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180302 05:04:27,274 EST INFO workflow.TransactionalWorkflowOperation [PostOptPortfolioScores_10027886_62087947442_62515891686_8008] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180302 05:04:27,278 EST INFO workflow.TransactionalWorkflowOperation [WorkflowManager_10027886_52909311282_52946733021_4000] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180302 05:04:27,282 EST INFO workflow.TransactionalWorkflowOperation [ClientPortfolioInitializationService_10027886_35383068482_35385507500_2700] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180305 06:10:43,829 EST INFO workflow.TransactionalWorkflowOperation [PreOptPortfolioScores_10027887_55858397233_55868440408_8001] - ---------- Committing all transactions using (workflowTransaction) transaction manager ----------
20180305 06:10:43,848 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499544} Connection
20180305 06:10:43,848 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499544} Preparing Call: { call SA_TAO_AIP_PORTFOLIO_PKG.GET_ALL_INV_PORTF_FOR_CLIENT(?, ?) }
20180305 06:10:43,848 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499545} Executing Statement: { call SA_TAO_AIP_PORTFOLIO_PKG.GET_ALL_INV_PORTF_FOR_CLIENT(?, ?) }
20180305 06:10:43,848 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499545} Parameters: [55858397233]
20180305 06:10:43,848 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499545} Types: [java.lang.Long]
20180305 06:10:43,856 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499546} Connection
20180305 06:10:43,856 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499546} Preparing Call: { call SA_TAO_AIP_OPT_TEMPLATE_PKG.get_portf_strtgy_ovrrde(?,?) }
20180305 06:10:43,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499547} Executing Statement: { call SA_TAO_AIP_OPT_TEMPLATE_PKG.get_portf_strtgy_ovrrde(?,?) }
20180305 06:10:43,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499547} Parameters: [55868440408]
20180305 06:10:43,856 EST DEBUG sql.PreparedStatement [WorkflowManager_10027887_55858397233_55868440408_2000] - {pstm-11499547} Types: [java.lang.Long]
20180305 06:10:43,867 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499548} Connection
20180305 06:10:43,867 EST DEBUG sql.Connection [WorkflowManager_10027887_55858397233_55868440408_2000] - {conn-11499548} Preparing Call: { call sa_tao_aip_portf_metric_pkg.get_portf_metric_val(?, ?, ?) }
... View more
03-04-2018
12:26 AM
Im trying to filter out events based on regex and index the remaining events based on below configs..But it doesn't seem to work...Can someone pls help..
In props.conf
[sourcetypename]
TRANSFORMS-set= setnull,setparsing
In transforms.conf
[setnull]
REGEX = (setting all transactions (.) transaction cases)|(Types:\s[.])|(FindingCall)|(Clearing junk and context)
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
... View more
01-23-2018
04:15 AM
Hi @rdagan,
When I run any job on the Hadoop server it works fine without any issues.
However it fails on the splunk Analytics for Hadoop Server.
I'm quite confused with the kerberos settings in indexes.conf for which I might need some help here:
I have splunkd1@TS.fitco.com Principal on the splunk analytics node and splunkd1@RT.rtp.com principal on hadoop cluster. Both these two different accounts with diff passwords.
I was able to run a mapreduce job on the Hadoop cluster using splunkd1@RT.rtp.com user account.
I generated keytab file (splunkd1.keytab) for splunkd1@RT.rtp.com on the Splunk node - at /home/splunkd1/ and updated indexes.conf with the same as below:
vix.kerberos.keytab = /home/splunkd1/splunkd1.keytab
vix.kerberos.principal = splunkd1@RT.rtp.com
Are the steps that I've followed incorrect or correct??
Kindly help.
... View more
12-16-2017
07:01 AM
Hi all
I have splunkd1@TS.fitco.com user on the splunk node and splunkd1@RT.rtp.com user on hadoop cluster.
I have created keytabfile for splunkd1@TS.fitco.com and provided in the indexes.conf....and I get this error while executing hadoop commands on splunk host.
ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "splunkdev@TS.fitco.com/xx.xx.xx.xxx"; destination host is: "SLPP02.HADOOP.company.COM:8020;
[hadoopidx]
coldPath = $SPLUNK_DB/hadoopidx/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/hadoopidx/db
maxTotalDataSizeMB = 20480
thawedPath = $SPLUNK_DB/hadoopidx/thaweddb
[provider:eihadoop]
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-hy2.jar
vix.dfs.namenode.kerberos.principal = hdfs/_HOST@HADOOP.company.COM
vix.env.HADOOP_HOME = /opt/local/hadoop-2.6.0-cdh5.9.1
vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.6.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-exec-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-metastore-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-serde-1.2.1.jar
vix.env.JAVA_HOME = /usr/java/jdk1.8.0_102
vix.family = hadoop
vix.fs.default.name = hdfs://SLPP02.HADOOP.company.COM:8020
vix.hadoop.security.authentication = kerberos
vix.hadoop.security.authorization = 1
vix.javaprops.java.security.krb5.kdc = SLP013.HADOOP.company.COM
vix.javaprops.java.security.krb5.realm = HADOOP.company.COM
vix.mapreduce.framework.name = yarn
vix.output.buckets.max.network.bandwidth = 0
vix.splunk.home.hdfs = /user/splunkdev/hadoopanalytics/
vix.yarn.nodemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.address = https://SLPP08.HADOOP.company.COM:8090/cluster
vix.yarn.resourcemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.scheduler.address = https://SLPP015.HADOOP.company.COM:8090/cluster/scheduler
vix.mapreduce.jobtracker.kerberos.principal = mapred/_HOST@HADOOP.company.COM
vix.kerberos.keytab = /home/splunkd1/splunkd1.keytab
vix.kerberos.principal = splunkdev@TS.company2.COM
[splunk_index_archive]
vix.output.buckets.from.indexes = hadoopidx
vix.output.buckets.older.than = 172800
vix.output.buckets.path = /user/splunkdev/splunk_index_archive
vix.provider = eihadoop
... View more
12-12-2017
03:40 AM
Hi @rdagan ,
I have splunkd1@TS.fitco.com user on the splunk node and splunkd1@RT.rtp.com user on hadoop cluster.
I have created keytabfile for splunkd1@TS.fitco.com and provided in the indexes.conf....and I get this error while executing hadoop commands on splunk host.
ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "splunkdev@TS.company2.COM/xx.xx.xx.xxx"; destination host is: "SLPP02.HADOOP.company.COM:8020;
[hadoopidx]
coldPath = $SPLUNK_DB/hadoopidx/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/hadoopidx/db
maxTotalDataSizeMB = 20480
thawedPath = $SPLUNK_DB/hadoopidx/thaweddb
[provider:eihadoop]
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-hy2.jar
vix.dfs.namenode.kerberos.principal = hdfs/_HOST@HADOOP.company.COM
vix.env.HADOOP_HOME = /opt/local/hadoop-2.6.0-cdh5.9.1
vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.6.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-exec-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-metastore-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-serde-1.2.1.jar
vix.env.JAVA_HOME = /usr/java/jdk1.8.0_102
vix.family = hadoop
vix.fs.default.name = hdfs://SLPP02.HADOOP.company.COM:8020
vix.hadoop.security.authentication = kerberos
vix.hadoop.security.authorization = 1
vix.javaprops.java.security.krb5.kdc = SLP013.HADOOP.company.COM
vix.javaprops.java.security.krb5.realm = HADOOP.company.COM
vix.mapreduce.framework.name = yarn
vix.output.buckets.max.network.bandwidth = 0
vix.splunk.home.hdfs = /user/splunkdev/hadoopanalytics/
vix.yarn.nodemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.address = https://SLPP08.HADOOP.company.COM:8090/cluster
vix.yarn.resourcemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.scheduler.address = https://SLPP015.HADOOP.company.COM:8090/cluster/scheduler
vix.mapreduce.jobtracker.kerberos.principal = mapred/_HOST@HADOOP.company.COM
vix.kerberos.keytab = /home/splunkd1/splunkd1.keytab
vix.kerberos.principal = splunkdev@TS.company2.COM
[splunk_index_archive]
vix.output.buckets.from.indexes = hadoopidx
vix.output.buckets.older.than = 172800
vix.output.buckets.path = /user/splunkdev/splunk_index_archive
vix.provider = eihadoop
... View more
12-07-2017
05:46 AM
Hi @rdagan ,
Thankyou for pointing out my mistakes...
I did make few errors..we have a HA CDH Cluster so I modified my below configs:
[provider:eihadoop]
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-hy2.jar
vix.dfs.namenode.kerberos.principal = hdfs/_HOST@HADOOP.company.COM
vix.env.HADOOP_HOME = /user/splunkdev
vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.6.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-exec-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-metastore-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-serde-1.2.1.jar
vix.env.JAVA_HOME = /usr/java/jdk1.8.0_102
vix.family = hadoop
vix.fs.default.name = hdfs://nameservice1
vix.yarn.resourcemanager.ha.rm-ids = rm570,rm780
vix.yarn.resourcemanager.address.rm570 = A.company.com:8032
vix.yarn.resourcemanager.address.rm780 = B.company.com:8032
vix.yarn.resourcemanager.scheduler.address.rm570 = A.company.com:8030
vix.yarn.resourcemanager.scheduler.address.rm780 = B.company.com:8030
vix.yarn.resourcemanager.ha.enabled = true
vix.yarn.resourcemanager.cluster-id = yarnRM
vix.yarn.application.classpath = $HADOOP_CLIENT_CONF_DIR,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/,$HADOOP_COMMON_HOME/lib/,$HADOOP_HDFS_HOME/,$HADOOP_HDFS_HOME/lib/,$HADOOP_YARN_HOME/,$HADOOP_YARN_HOME/lib/
vix.hadoop.security.authentication = kerberos
vix.hadoop.security.authorization = 1
vix.javaprops.java.security.krb5.kdc = SLP013.HADOOP.company.COM
vix.javaprops.java.security.krb5.realm = HADOOP.company.COM
vix.mapreduce.framework.name = yarn
vix.output.buckets.max.network.bandwidth = 0
vix.splunk.home.hdfs = /user/splunkdev/hadoopanalytics/
vix.yarn.nodemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.mapreduce.jobtracker.kerberos.principal = mapred/_HOST@HADOOP.company.COM
vix.kerberos.keytab = /export/home/splunkdev/splunkdev.keytab
vix.kerberos.principal = splunkdev@TSS.company.COM
However Im unable to run hadoop commands on my splunk host:
bash-4.1$ hdfs dfs -ls /
17/12/07 03:04:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.ExceptionInInitializerError
...................................................................................
Caused by: java.lang.RuntimeException: Bailing out since native library couldn't be loaded
at org.apache.hadoop.security.JniBasedUnixGroupsMapping. (JniBasedUnixGroupsMapping.java:46)
... 30 more
Since I quite new to cloudera , Kindly clarify my below doubts:
1. I copied all config files manually into the splunk server.
Since splunk is a normal-Hadoop client , does that mean the splunk libraries needs to installed on the Edge-Node that the CDH cluster has?
Do I need to get Cloudera Client parcels on teh splunk host? If yes, how Do I get them into the splunk server? Does it required to be added in the cloudera manager?
... View more
12-02-2017
12:48 AM
Sorry thats a typo...
Yes I used the same.
/opt/hadoop-2.6.0-cdh5.9.1/bin/hdfs dfs -ls hdfs://SLPP02.HADOOP.company.com:8020/ /user/splunkdevuser/
Should I update/add any Hadoop configuration files (core-site.xml etc..) on the splunk servers as well?
... View more
12-01-2017
06:18 AM
Im trying to send splunk-index data to Hadoop using Hadoop Data roll.
However Im not able to establish connection between splunk and Hadoop at all...I get below error on my splunk indexer
bash-4.1$ /opt/hadoop-2.6.0-cdh5.9.1/bin/hdfs dfs -ls hdfs://hadoopnamenode.company.com:8020/ /user/splunkdevuser/
17/12/01 08:38:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
ls: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
Warning: fs.defaultFS is not set when running "ls" command.
ls: `/user/splunkd1/': No such file or directory
Can someone kindly help
Below is my indexes.conf
[hadoopidx]
coldPath = $SPLUNK_DB/hadoopidx/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/hadoopidx/db
maxTotalDataSizeMB = 20480
thawedPath = $SPLUNK_DB/hadoopidx/thaweddb
[provider:eihadoop]
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-hy2.jar
vix.dfs.namenode.kerberos.principal = hdfs/_HOST@HADOOP.company.COM
vix.env.HADOOP_HOME = /user/splunkdev
vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.6.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-exec-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-metastore-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-serde-1.2.1.jar
vix.env.JAVA_HOME = /usr/java/jdk1.8.0_102
vix.family = hadoop
vix.fs.default.name = hdfs://SLPP02.HADOOP.company.COM:8020
vix.hadoop.security.authentication = kerberos
vix.hadoop.security.authorization = 1
vix.javaprops.java.security.krb5.kdc = SLP013.HADOOP.company.COM
vix.javaprops.java.security.krb5.realm = HADOOP.company.COM
vix.mapreduce.framework.name = yarn
vix.output.buckets.max.network.bandwidth = 0
vix.splunk.home.hdfs = /user/splunkdev/hadoopanalytics/
vix.yarn.nodemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.address = https://SLPP08.HADOOP.company.COM:8090/cluster
vix.yarn.resourcemanager.principal = yarn/_HOST@HADOOP.company.COM
vix.yarn.resourcemanager.scheduler.address = https://SLPP015.HADOOP.company.COM:8090/cluster/scheduler
vix.mapreduce.jobtracker.kerberos.principal = mapred/_HOST@HADOOP.company.COM
vix.kerberos.keytab = /export/home/splunkdev/splunkdev.keytab
vix.kerberos.principal = splunkdev@TSS.company.COM
[splunk_index_archive]
vix.output.buckets.from.indexes = hadoopidx
vix.output.buckets.older.than = 172800
vix.output.buckets.path = /user/splunkdev/splunk_index_archive
vix.provider = eihadoop
... View more
12-01-2017
02:33 AM
HI @rdaga ,
vix.kerberos.principal =
vix.kerberos.keytab =
Does this principal and keytab refer to the user that I create on Hadoop cluster?
/user/splunkuser/ in Hadoop?
... View more
11-20-2017
05:41 AM
As per the below doc to set up kerberos authentication for hadoop:
https://docs.splunk.com/Documentation/Splunk/7.0.0/HadoopAnalytics/ConfigureKerberosauthentication
To which realm does the below parameter refer to? Our hadoop cluster is in a different realm and splunk cluster is ina different realm. Whose realm details should I provide here?
vix.java.security.krb5.kdc =
*vix.java.security.krb5.realm = *
... View more
11-07-2017
11:51 PM
HI @somesoni2,
When I export the results I dont get all the folders for the host. Only the first folder for any host is present.
Why does this happen? KindLy help.
... View more
11-07-2017
11:49 PM
HI @niketnilay ,
When I export the results I dont get all the folders for the host. Only the first folder for any host is present.
Why does this happen? KindLy help.
... View more
11-07-2017
03:13 AM
I have a statistics table-panel in my dashboard.
By the default each page displays some random no of evets and it's really long(the panel) in my dashboard.
Can we restrict the no of events/rows in a page in the panel?
I tried looking for help and limit was mentioned. But that would restrict the entire result.
I dont want that. Its just that the events should be displayed in more pages but not long panels.
Hope my Qs is not vague.
Kindly suggest
... View more
11-06-2017
08:50 AM
1.
I have many directories of the below pattern in a set of hosts:
/opt/ab/admin/abdc/apache/main.logs
/opt/ab/admin/xyzz/apache/main.logs
I want to tabulate all the foldernames i.e the 4th sub-dir in a host like below:
host foldername
A abcd
xyzz
B abcd
pqrs
xyzz
Is there any way I can achieve this in Splunk?
I tried to execute a shell script that just lists these folders names and this is available in the events as shown in screenshot attached. But Im not sure how I can tabulate them for the host?
Kindly provide suggestions
... View more
11-05-2017
06:46 AM
Thnakyou @ kamlesh_vaghela , that helped.
... View more
11-04-2017
08:40 AM
I have a dashboard for showing the status of hosts.
I want the dashboard to populate when I choose the hosts via drop down/ radiobutton.
The two hosts groups are something like this.
host=ab* OR host=cd*
host=xy* OR host=un*
My dashboard panels have a search like below:
index = abc hosts=ab* OR host =bc* source =svd.log....
Although I can create a dropdown for these host types I dont think i can replace the highlighted value in the query...
Can someone suggest me how to achieve this>?
... View more
11-04-2017
01:34 AM
HI @pyro_wood ,
Thankyou for the help..
My logfile sometimes contains like this
/opt/pq/logs/xyzz.backup1213/apache/pqrs-tcm.log
How can I modify the expression to extrat only xyzz without the the word that follow '.' operator? like in above example
... View more
11-03-2017
04:10 AM
I have many sources/logfiles in a host like this:
/opt/ab/logs/abcd/apache/abcd-tcm.log
/opt/xy/logs/xyzz/apache/xyzz-tcm.log
/opt/pq/logs/xyzz/apache/pqrs-tcm.log
Im interested in extracting the third string of the log and I tried it via the below command to MyFieldName.
rex field=source "(\/opt\/.\/logs\/(?.)\/apache\/.)"
Now I want to table out these MyFieldName for a list of hosts. How can I achieve this?
ie. Host A might have MyFieldName 4 values.
Host B misht MyFieldName 3 values.
These MyFieldName can be common amongst the hosts.
Im not able to get this ont-to-many (server-to-MyFieldName values) Table.
When I try to dedup host OR MyFieldName with belwo search , it truncates the results.
index="capgm" sourcetype=tc host=* | rex field=source "(\/opt\/.\/logs\/(?.)\/apache\/.)" | where MyFieldName like "%%" | rename MyFieldName AS NewField, host AS SERVER | table NewField, SERVER | dedup NewField
I want something like this
HOST MyFieldName
A xyzz
abcd
B xyzz
pqrs
C xyzz
abcd
pqrs
... View more
10-23-2017
06:16 AM
Thankyou .
@Damien Dallimore ,
Does this app support Kafka as well? Can it consume messages from Kafka?
... View more
10-17-2017
03:08 AM
Where do I install this app?
On SH Cluster or Heavy Forwarder via deployment server?
I don't find any splunk dcoumentation on this..
Can some one please share teh link of documentation on this?
... View more
