We have a requirement to enable TLS on traffic from a universal forwarder (UF) to a heavy forwarder. We will be using self-signed certificates for this. From the following Splunk documentation, we understand how to generate and apply certificates for heavy forwarders. We are not clear on how to generate certificates for UF (client.pem). Splunk Universal Forwarder - v6.1.7.1 Splunk Enterprise - v6.6.5 Can you please point me in the right direction for generating client certificates. https://docs.splunk.com/Documentation/Splunk/7.2.0/Security/Howtoself-signcertificates http://docs.splunk.com/Documentation/Splunk/7.2.0/Security/HowtoprepareyoursignedcertificatesforSplunk http://docs.splunk.com/Documentation/Splunk/7.2.0/Security/ConfigureSplunkforwardingtousesignedcertificates ... View more

I have a search head cluster with 6 nodes + 1 deployer with Splunk 6.3.1. Is there a dependency for the deployer to be up and running always for the search head cluster to work? There is a planned infra activity on the deployer to be down for 24 hours. Will there be an impact on the search head cluster when deployer is down (except no new deployments can be done during this period)? ... View more

I have added the PROD and DR indexer hosts using add search-server CLI command. Now my requirement is to keep the PROD indexers as enabled and DR indexers as disabled. Is there a CLI command to achieve this? Do I need to disable via UI only? ... View more

Hi, Can someone please point out what am I missing below? I am not receiving the scheduled pdf dashboard delivery in my inbox. No errors reported in python.log, scheduler.log and splunkd.log. I am using Splunk 6.0.5. Note: I am able to receive the scheduled saved search results in csv format in my inbox. So my mailserver is working fine. Below is the snapshot of alert_actions.conf [default] hostname = ttl = 86400 command = sendemail "to=$action.email.to$" "server=$action.email.mailserver{default=localhost}$" "from=$action.email.from{default=splunk@localhost}$" "subject=$action.email.subject{recurse=yes}$" "format=$action.email.format{default=csv}$""sssummary=Saved Search [$name$]: $counttype$($results.count$)" "sslink=$results.url$" "ssquery=$search$" "ssname=$name$" "inline=$action.email.inline{default=False}$" "sendresults=$action.email.sendresults{default=False}$" "sendpdf=$action.email.sendpdf{default=False}$" "pdfview=$action.email.pdfview$" "searchid=$search_id$" "graceful=$graceful{default=True}$" maxinputs="$maxinputs{default=1000}$" maxtime="$action.email.maxtime{default=5m}$"_validate-1 = action.email.sendresults, validate( is_bool('action.email.sendresults'), "Value of argument 'action.email.sendresults' must be a boolean") [email] subject = $name$:~:$description$ from = myname@mycompany.com mailserver = smtprelay.mycompany.com sendpdf = 1 reportServerEnabled = false reportServerURL = reportPaperSize = a4 reportPaperOrientation = portrait reportIncludeSplunkLogo = 1 sendresults = 1 ... View more

I get the following error in jbridge.log. Can someone advice on the fix. 2015-01-13 08:21:11,348 DEBUG options=['-Xmx512m', '-Dfile.encoding=UTF-8', '-server', '-Duser.language=en', '-Duser.region=', '-Dsplunk.app.ctx=dbx'] 2015-01-13 08:21:11,348 INFO Checking for obsolete java libraries in /apps/splunk/etc/apps/dbx/bin/lib 2015-01-13 08:21:11,348 DEBUG Starting JavaBridgeServer... 2015-01-13 08:21:11,349 DEBUG Executing (native) CMD /usr/java/jdk-1.6.0_26-x86_64/bin/java -cp /apps/splunk/etc/apps/dbx/bin/lib/jdbm-2.2.jar:/apps/splunk/etc/apps/dbx/bin/lib/jtds-1.2.6.jar:/apps/splunk/etc/apps/dbx/bin/lib/dbx.jar:/apps/splunk/etc/apps/dbx/bin/lib/h2-1.3.162.jar:/apps/splunk/etc/apps/dbx/bin/lib/antlr-2.7.7.jar:/apps/splunk/etc/apps/dbx/bin/lib/commons-pool-1.5.6.jar:/apps/splunk/etc/apps/dbx/bin/lib/commons-logging-1.0.4.jar:/apps/splunk/etc/apps/dbx/bin/lib/log4j-1.2.15.jar:/apps/splunk/etc/apps/dbx/bin/lib/postgresql-9.0-801.jdbc3.jar:/apps/splunk/etc/apps/dbx/bin/lib/stringtemplate-3.2.1.jar:/apps/splunk/etc/apps/dbx/bin/lib/xstream-1.4.1.jar:/apps/splunk/etc/apps/dbx/bin/lib/hsqldb.jar:/apps/splunk/etc/apps/dbx/bin/lib/sqlite-jdbc-3.7.2.jar -Xmx512m -Dfile.encoding=UTF-8 -server -Duser.language=en -Duser.region= -Dsplunk.app.ctx=dbx com.splunk.bridge.JavaBridgeServer 13747 2015-01-13 08:21:11,349 INFO Started JavaBridgeServer PID=13779 2015-01-13 08:21:11,350 DEBUG starting pid watcher... 2015-01-13 08:21:12,422 ERROR Java process returned error code 1! Error: Initializing Splunk context... Environment: SplunkEnvironment{SPLUNK_HOME=/apps/splunk,SPLUNK_DB=/apps/splunk/var/lib/splunk} Configuring Log4j... Exception in thread "main" com.splunk.config.SplunkConfigurationException: IO Error while reading configuration from Splunkd: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000 at com.splunk.config.rest.RESTAdapter.request(RESTAdapter.java:195) at com.splunk.config.rest.RESTAdapter.readConfig(RESTAdapter.java:203) at com.splunk.config.cache.CachedConfigurationAdapter.readConfig(CachedConfigurationAdapter.java:32) at com.splunk.config.cache.CachedConfigurationAdapter.readStanza(CachedConfigurationAdapter.java:40) at com.splunk.env.SplunkContext.getConfigStanza(SplunkContext.java:313) at com.splunk.env.SplunkContext.initialize(SplunkContext.java:128) at com.splunk.bridge.JavaBridgeServer.main(JavaBridgeServer.java:34) Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1491) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1485) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1139) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:234) at com.splunk.rest.Response.getContent(Response.java:49) at com.splunk.rest.Response.readXML(Response.java:57) at com.splunk.config.rest.RESTAdapter.request(RESTAdapter.java:193) ... 6 more Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318) at com.splunk.rest.Response.(Response.java:26) at com.splunk.rest.Splunkd.request(Splunkd.java:208) at com.splunk.rest.Splunkd.request(Splunkd.java:98) ... 7 more 2015-01-13 08:21:12,422 ERROR Command output: None 2015-01-13 08:21:12,422 DEBUG Error waiting for process: Java process returned error code 1! Error: Initializing Splunk context... Environment: SplunkEnvironment{SPLUNK_HOME=/apps/splunk,SPLUNK_DB=/apps/splunk/var/lib/splunk} Configuring Log4j... Exception in thread "main" com.splunk.config.SplunkConfigurationException: IO Error while reading configuration from Splunkd: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000 at com.splunk.config.rest.RESTAdapter.request(RESTAdapter.java:195) at com.splunk.config.rest.RESTAdapter.readConfig(RESTAdapter.java:203) at com.splunk.config.cache.CachedConfigurationAdapter.readConfig(CachedConfigurationAdapter.java:32) at com.splunk.config.cache.CachedConfigurationAdapter.readStanza(CachedConfigurationAdapter.java:40) at com.splunk.env.SplunkContext.getConfigStanza(SplunkContext.java:313) at com.splunk.env.SplunkContext.initialize(SplunkContext.java:128) at com.splunk.bridge.JavaBridgeServer.main(JavaBridgeServer.java:34) Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1491) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1485) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1139) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:234) at com.splunk.rest.Response.getContent(Response.java:49) at com.splunk.rest.Response.readXML(Response.java:57) at com.splunk.config.rest.RESTAdapter.request(RESTAdapter.java:193) ... 6 more Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318) at com.splunk.rest.Response.(Response.java:26) at com.splunk.rest.Splunkd.request(Splunkd.java:208) at com.splunk.rest.Splunkd.request(Splunkd.java:98) ... 7 more 2015-01-13 08:21:12,422 DEBUG JavaBridgeServer terminated ... View more

The below is the windows security logs Message field data. The Security_ID field is splunk identified and contains 2 values - Domain1\username1 and Domain2\username2. I want to extract them in search query as 2 different fields using regex say myfield1=Domain1\username1 and myfield2=Domain2\username2. Any pointers on how to write regex for this would be helpful? I used the below query index="win_logs" sourcetype="WinEventLog:Security" (EventCode=4720) | table Message "A user account was created. Subject: Security ID: Domain1\username1 Account Name: username1 Account Domain: Domain1 Logon ID: 0x1005dc243 New Account: Security ID: Domain2\username2 Account Name: username2 Account Domain: Domain2 Attributes: SAM Account Name: username2 Display Name: first name, last name User Principal Name: - Home Directory: - Home Drive: - Script Path: - Profile Path: - User Workstations: - Password Last Set: Account Expires: Primary Group ID: 513000000 Allowed To Delegate To: - Old UAC Value: 0x0 New UAC Value: 0x15 User Account Control: Account Disabled 'Password Not Required' - Enabled 'Normal Account' - Enabled User Parameters: - SID History: - Logon Hours: Additional Information: Privileges -" ... View more