About splunkIT

splunkIT · ‎12-19-2013

Hello, I am having a hard time trying to pin down why most of my real-time alerts have stopped working. I have looked into scheduler.log and python.log, and did not find much insightful details to the problem. Here are the symptoms: Only real-time alerts are not appearing to fire Non-realtime alerts appear to be fine as I am still getting alert emails Once splunk is restarted, some rt alerts appear to be firing; then eventually stopped

splunkIT · ‎11-19-2013

jbsplunk is a badass, and you rock!!

splunkIT · ‎11-18-2013

At the DB Connect source code level it seems the developer is attempting to create an SSL connection as. SSLContext sc = SSLContext.getInstance("SSL"); My org doesn’t allow SSL so the communication fails between the java bridge server and client. It has to use TLS for it to work on our systems. SSLContext sc = SSLContext.getInstance("TLS");

splunkIT · ‎11-18-2013

JavaBridge Server/Client communication fails with the following error (stack): 2013-11-12 17:26:36,294 INFO Checking for obsolete java libraries in /opt/splunk/etc/apps/dbx/bin/lib 2013-11-12 17:26:36,295 INFO Started JavaBridgeServer PID=10643 2013-11-12 17:26:36,708 ERROR Java process returned error code 1! Error: Initializing Splunk context... Environment: SplunkEnvironment{SPLUNK_HOME=/opt/splunk,SPLUNK_DB=/opt/splunk/var/lib/splunk} Configuring Log4j... Exception in thread "main" java.lang.ExceptionInInitializerError at com.splunk.env.SplunkContext.createRestService(SplunkContext.java:148) at com.splunk.env.SplunkContext.initialize(SplunkContext.java:106) at com.splunk.bridge.JavaBridgeServer.main(JavaBridgeServer.java:34) Caused by: com.splunk.rest.SplunkdException: Error while disabling SSL certificate verification: java.security.NoSuchAlgorithmException: SSL SSLContext not available at com.splunk.rest.Splunkd.createSSLFactory(Splunkd.java:346) at com.splunk.rest.Splunkd. (Splunkd.java:51) ... 3 more Caused by: java.security.NoSuchAlgorithmException: SSL SSLContext not available at sun.security.jca.GetInstance.getInstance(GetInstance.java:142) at javax.net.ssl.SSLContext.getInstance(SSLContext.java:125) at com.splunk.rest.Splunkd.createSSLFactory(Splunkd.java:342) ... 4 more 2013-11-12 17:26:36,708 ERROR Command output: None

splunkIT · ‎11-15-2013

In that Database Input view: Splunk>Manager>>Data>>Data Inputs>>new, under the Tail Input - Rising Column field, there is a brief explanation: " Choose a column with an increasing value. Such as a creation or modification timestamp or a sequential identifier. You can also create a trigger to synthesize such a value." What is intended by "create a trigger to synthesize such a value"? Is it referring to something that can be done in Splunk, or in our database?

splunkIT · ‎11-11-2013

DB Connect 1.1.1 or later should have addressed this problem. I have also seen this issue with java 1.6. You should also upgrade your java to the latest 1.7 version: http://www.oracle.com/technetwork/java/javase/downloads/index.html Better yet, consider using the java sdk package, because Support may request for jstack outputs if in-depth troubleshooting is required: http://answers.splunk.com/answers/102822/how-to-generate-java-jstack-of-the-db-connect-jbridge-process

splunkIT · ‎11-07-2013

Updated docs link: http://docs.splunk.com/Documentation/Splunk/6.0/Security/ConfigureSplunkforwardingtousesignedcertificates http://docs.splunk.com/Documentation/Splunk/latest/Security/AboutsecuringyourSplunkconfigurationwithSSL http://docs.splunk.com/Documentation/Splunk/latest/Security/AboutsecuringyourSplunkconfigurationwithSSL#Getting_your_certificates

splunkIT · ‎10-10-2013

Generally splunk would delete the file one they have been indexed. Can you confirm whether the files have already been indexed? Also check the splunkd.log for relevant errors.

splunkIT · ‎09-17-2013

In order to obtain jstack output on a java process, you would first need to obtain the java sdk package. The standard jre package does not include the necessary tools (ie. jps and jstack) for generating the jstack output. Requirements: 1.) Obtain java sdk from http://www.oracle.com/technetwork/java/javase/downloads/index.html 2.) Reconfigure db connect to use the sdk version of the jvm. For example, if you specify /usr/lib/jvm/jdk1.7.0_25 as the java home, ensure that there is a bin sub-directory which contains the java, jps, and jstack executables. While the issue is occurring: 1.) Open a command prompt and cd into the $JAVA_HOME/bin directory 2.) Run: jps -l (this will list running java processes) 3.) Identify the process with the label com.splunk.bridge.JavaBridgeServer and copy the PID (number) 4.) Run: jstack -l (where is replaced with the copied PID from step #3) 5.) Copy the output and paste it into a text file for Support

splunkIT · ‎09-13-2013

Splunk doesn't search for libeay32.dll, the host system does (i.e. image activation). This could probably be solved by modifying the PATH environment variable; which is a search list. For Splunk, make sure the search path looks in the splunk directory first.

splunkIT · ‎09-11-2013

Example for Windows 2008 WinSecurity events: [dropevents] REGEX = (?msi)^EventCode=(4776|4648|4624|4634).*^Keywords=Audit\sSuccess DEST_KEY = queue FORMAT = nullQueue You can also use the following site to verify the regex: http://gskinner.com/RegExr/?31r9a

splunkIT · ‎09-04-2013

By adding this -Djsse.enableCBCProtection=false param in the java options, I can connect to SSL enabled MSSQL instance without having to add any new parameters in the JDBC connection URL. This appears to be a java bug: https://www.aquaclusters.com/app/home/project/public/aquadatastudio/discussion/GeneralDiscussions/post/25/Java-6u29-bug-prevents-SSL-connection-to-SQL-Server-2008-R2 https://forums.oracle.com/message/10843957 To workaround it, I have to add "-Djsse.enableCBCProtection=false" to the java options parameter, by appending it in the $SPLUNK_HOME/apps/dbx/local/java.conf file. Here is mine for your reference: [java] home = /home/luan/jdk1.7.0_25 options = -Xmx256m -Dfile.encoding=UTF-8 -server -Djsse.enableCBCProtection=false -Duser.language=en -Duser.reg ion=

splunkIT · ‎09-04-2013

My DB Connect app has no problem connecting to a non-ssl enforced MSSQL instance. However, when connecting to a ssl enforced MSSQL instance, db connect appears to hang. One interesting observation is that the problem (hang) seems to be reproducible using a 3rd party app like DBVisualizer to connect using the jTDS driver (same driver use by DB Connect).

splunkIT · ‎08-20-2013

It turns out that the role SPLNK_DNS is restricted to search terms "index=dns_index". Once this restriction is removed from Manager » Access controls » Roles » SPLNK_DNS "Restrict search terms" field, user belonging to to role SPLNK_DNS can now search other indexes.

splunkIT · ‎08-16-2013

AD user is assigned to role SPLNK_DNS which is working fine. User also needs to be part of other role called exchange. Splunk Manager UI shows that user is part of both roles and no erros in AD logs when user logs it. However, when he searches for exhange logs, no events shows even though events are there. Also noticed that if role SPLUNK_DNS is removed from this user, he is able to search the the exchange datatype with results.

splunkIT · ‎06-28-2013

First thing to check would be to review the dbx.log and jbridge.log files in $SPLUNK_HOME/var/log/splunk for any relevant error. You can also enable DEBUG logging by adding the following entries in the $SPLUNK_HOME/etc/apps/dbx/local/java.conf: [logging] level = DEBUG A splunkd restart would be required after the above changes. The DEBUG info will be written in the dbx.log as well. Otherwise, further troubleshooting of the issue may require gathering java thread dump to be submitted to Splunk Support for further investigation. FYI: There is also a known bug (DBX-151) reported for DB Connect 1.0.10 with similar symptoms.

splunkIT · ‎06-17-2013

Does anyone know how to: 1) search for which user has what access to the index? 2) who has accessed to what index within like the last 24 hours?

splunkIT · ‎06-10-2013

Here is an example of the [accepted_keys] usage: http://splunk-base.splunk.com/answers/90930/upgraded-to-splunk-503-and-noticing-undocumented-key-used-in-transformsconf-messages-during-startup

splunkIT · ‎06-10-2013

This is a known bug (SPL-68932) in Splunk 5.0.3. The message is rather harmless, and your _SYSLOG_ROUTING should still works as usual. You can either ignore the message during splunk startup, or by adding the following entries in your transforms.conf to make the message go away: [accepted_keys] is_valid=_SYSLOG_ROUTING More details on this [accepted_keys] stanza here: http://docs.splunk.com/Documentation/Splunk/5.0.3/Admin/Transformsconf Once you have made the above changes and restart splunk, the warning messages should go away.

splunkIT · ‎06-10-2013

According to http://docs.splunk.com/Documentation/Splunk/5.0.3/Admin/Transformsconf [accepted_keys] = Modifies Splunk's list of key names it considers valid when automatically checking your transforms for use of undocumented SOURCE_KEY or DEST_KEY values in index-time transformations. By adding entries to [accepted_keys], you can tell Splunk that a key that is not documented is a key you intend to work for reasons that are valid in your environment / app / etc. The 'name' element is simply used to disambiguate entries, similar to -class entries in props.conf. The name can be anything of your chosing, including a descriptive name for why you use the key. The entire stanza defaults to not being present, causing all keys not documented just above to be flagged.

Posts	165
Solutions	23
Karma Given	189
Karma Received	199
Member Since	‎06-25-2012

Online Status	Offline
Date Last Visited	‎05-11-2021 06:10 PM

How to collect DEBUG search artifacts (specificall...

Why is a number getting appended to each line when...

Splunk Palo Alto Add-on 6.0.2 and Enterprise Secur...

KVStore mongo db failing to start, after kvstore i...

How to prevent a custom app from being deleted in ...

Index time extraction: How to field extract from s...

Deployment Server Whitelist help

How to include a few events from the log prior to...

User environment's PYTHONHOME variable is preventi...

Why is our Search Head Cluster captain generating ...

How to troubleshoot real-time alerts not working?

Re: Does CVE-2013-6771 require upgrading to minimu...

Re: JavaBridge server failed to start - java.secur...

JavaBridge server failed to start - java.security....

DB Connect: How to create a trigger to synthesize ...

Re: The Java Bridge server is not running DBX 1.10

Re: Splunk default SSL certificate expired for SSL...

Re: dbConnect sinkhole not working for dbmon direc...

Re: How to generate java jstack of the db connect ...

Re: splunk locking libeay32.dll from wrong directo...

Re: WinEventLog:Security filter

Re: DB Connect hangs when attempting to connect to...

DB Connect hangs when attempting to connect to MSS...

Re: User with multiple splunk roles cannot search ...

User with multiple splunk roles cannot search cert...

Re: dbmon-tail stanza has stopped processing data ...

How to search for which user has access to what in...

Re: What is [accepted_keys] in transforms.conf?

Re: Upgraded to Splunk 5.0.3, and noticing "Undocu...

Re: What is [accepted_keys] in transforms.conf?