Deployment Architecture
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question

ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.

ahmemohs03
Explorer
‎07-19-2018 09:33 AM

07-18-2018 21:20:40.725 +0000 WARN X509Verify - X509 certificate (O=SplunkUser,CN=SplunkServerDefaultCert) should not be used, as it is issued by Splunk's own default Certificate Authority (CA). This puts your Splunk instance at very high-risk of the MITM attack. Either commercial-CA-signed or self-CA-signed certificates must be used; see:
07-18-2018 21:20:40.736 +0000 INFO WatchedFile - Will begin reading at offset=392049 for file='/welldata/splunk/var/log/introspection/disk_objects.log'.
07-18-2018 21:20:40.740 +0000 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/welldata/splunk/var/log/introspection/http_event_collector_metrics.log'.
07-18-2018 21:20:40.799 +0000 ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.
07-18-2018 21:20:40.967 +0000 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/welldata/splunk/var/log/splunk/btool.log'.
07-18-2018 21:20:40.970 +0000 INFO WatchedFile - Will begin reading at offset=3894 for file='/welldata/splunk/var/log/splunk/splunkd-utility.log'.
07-18-2018 21:20:40.977 +0000 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/welldata/splunk/var/log/splunk/searchhistory.log'.
07-18-2018 21:20:40.981 +0000 INFO WatchedFile - Will begin reading at offset=238867 for file='/welldata/splunk/var/log/splunk/splunkd_access.log'.
07-18-2018 21:20:40.998 +0000 INFO WatchedFile - Will begin reading at offset=3141787 for file='/welldata/splunk/var/log/splunk/audit.log'.
07-18-2018 21:20:41.001 +0000 INFO WatchedFile - Will begin reading at offset=933 for file='/welldata/splunk/var/log/splunk/conf.log'.
07-18-2018 21:20:41.020 +0000 INFO WatchedFile - Will begin reading at offset=2076287 for file='/welldata/splunk/var/log/splunk/health.log'.
07-18-2018 21:20:43.337 +0000 INFO IntrospectionGenerator:resource_usage - RU_main - I-data gathering (Resource Usage) starting; period=10s
07-18-2018 21:20:43.349 +0000 INFO IntrospectionGenerator:resource_usage - RU_main - I-data gathering (IO Statistics) starting; interval=60s
07-18-2018 21:20:46.023 +0000 WARN TailReader - Could not send data to output queue (parsingQueue), retrying...
07-18-2018 21:20:48.590 +0000 INFO ExecProcessor - message from "python /welldata/splunk/etc/apps/splunk_monitoring_console/bin/dmc_config.py" Cannot detect SHC status because of License Restriction. Will not disable DMC.
07-18-2018 21:21:10.392 +0000 INFO ScheduledViewsReaper - Scheduled views reaper run complete. Reaped count=0 scheduled views
07-18-2018 21:52:08.785 +0000 WARN TcpInputProc - Stopping all listening ports. Queues blocked for more than 300 seconds
07-18-2018 21:52:08.785 +0000 INFO TcpInputProc - Stopping IPv4 port 9997
07-18-2018 21:59:59.999 +0000 INFO ExecProcessor - setting reschedule_ms=3600002, for command=python /welldata/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py
07-18-2018 23:00:00.003 +0000 INFO ExecProcessor - setting reschedule_ms=3599997, for command=python /welldata/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py
07-19-2018 00:00:00.000 +0000 INFO ExecProcessor - setting reschedule_ms=3600000, for command=python /welldata/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py
07-19-2018 00:00:00.000 +0000 INFO ExecProcessor - setting reschedule_ms=86400000, for command=python /welldata/splunk/etc/apps/splunk_instrumentation/bin/schedule_delete.py
07-19-2018 00:00:00.945 +0000 INFO LMStackMgr - should rollover=true because _lastRolloverTime=1531872000 lastRolloverDay=1531872000 snappedNow=1531958400
07-19-2018 00:00:00.945 +0000 INFO LMStackMgr - finished rollover, new lastRolloverTime=1531958400
07-19-2018 00:00:28.945 +0000 INFO LMSlaveInfo - Detected that masterTimeFromSlave(Wed Jul 18 23:59:27 2018) < lastRolloverTime(Thu Jul 19 00:00:00 2018), meaning that the master has already rolled over. Ignore slave persisted usage.
07-19-2018 01:59:59.999 +0000 INFO ExecProcessor - setting reschedule_ms=3600002, for command=python /welldata/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py
07-19-2018 03:00:00.001 +0000 INFO ExecProcessor - setting reschedule_ms=3599999, for command=python /welldata/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py
07-19-2018 03:01:00.599 +0000 WARN TelemetryHandler - 1531872000.000000

Tags (1)
0 Karma
Reply

auraria1
Path Finder
‎07-19-2018 12:22 PM

Good call,
@ahmemohs03 I'd run a chown on the entire directory for the dedicated splunk user,

chown splunk:splunk -R /opt/splunkforwarder/

Swap out splunk for the user running splunk.

0 Karma
Reply

ahmemohs03
Explorer
‎07-19-2018 11:58 AM

Hi Patil,

Yes, Splunk universal forwarder does not have web UI. I am saying about splunk enterprises this installed on Linux A machine and splunk forwarder on Linux B machine. Splunk B logs need to see forwarder on Linux A(Splunk web url).

output conf file of universal forwarder is located at : /opt/splunkforwarder/etc/system/local

after running run ./splunk btool --debug outputs list

/opt/splunkforwarder/etc/system/default/outputs.conf [syslog]
/opt/splunkforwarder/etc/system/default/outputs.conf maxEventSize = 1024
/opt/splunkforwarder/etc/system/default/outputs.conf priority = <13>
/opt/splunkforwarder/etc/system/default/outputs.conf type = udp
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf [tcpout]
/opt/splunkforwarder/etc/system/default/outputs.conf ackTimeoutOnShutdown = 30
/opt/splunkforwarder/etc/system/default/outputs.conf autoLBFrequency = 30
/opt/splunkforwarder/etc/system/default/outputs.conf autoLBVolume = 0
/opt/splunkforwarder/etc/system/default/outputs.conf blockOnCloning = true
/opt/splunkforwarder/etc/system/default/outputs.conf blockWarnThreshold = 100
/opt/splunkforwarder/etc/system/default/outputs.conf cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES256-SHA384:ECDH-ECDSA-AES128-SHA256
/opt/splunkforwarder/etc/system/default/outputs.conf compressed = false
/opt/splunkforwarder/etc/system/default/outputs.conf connectionTimeout = 20
/opt/splunkforwarder/etc/system/local/outputs.conf defaultGroup = default-autolb-group
/opt/splunkforwarder/etc/system/default/outputs.conf disabled = false
/opt/splunkforwarder/etc/system/default/outputs.conf dropClonedEventsOnQueueFull = 5
/opt/splunkforwarder/etc/system/default/outputs.conf dropEventsOnQueueFull = -1
/opt/splunkforwarder/etc/system/default/outputs.conf ecdhCurves = prime256v1, secp384r1, secp521r1
/opt/splunkforwarder/etc/system/default/outputs.conf forceTimebasedAutoLB = false
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf forwardedindex.0.whitelist = .*
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf forwardedindex.1.blacklist = _.*
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf forwardedindex.2.whitelist = (_audit|_introspection|_internal|_telemetry)
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf forwardedindex.filter.disable = false
/opt/splunkforwarder/etc/system/default/outputs.conf heartbeatFrequency = 30
/opt/splunkforwarder/etc/system/default/outputs.conf indexAndForward = false
/opt/splunkforwarder/etc/system/default/outputs.conf maxConnectionsPerIndexer = 2
/opt/splunkforwarder/etc/system/default/outputs.conf maxFailuresPerInterval = 2
/opt/splunkforwarder/etc/system/default/outputs.conf maxQueueSize = auto
/opt/splunkforwarder/etc/system/default/outputs.conf readTimeout = 300
/opt/splunkforwarder/etc/system/default/outputs.conf secsInFailureInterval = 1
/opt/splunkforwarder/etc/system/default/outputs.conf sendCookedData = true
/opt/splunkforwarder/etc/system/default/outputs.conf sslQuietShutdown = false
/opt/splunkforwarder/etc/system/default/outputs.conf sslVersions = tls1.2
/opt/splunkforwarder/etc/system/default/outputs.conf tcpSendBufSz = 0
/opt/splunkforwarder/etc/system/default/outputs.conf useACK = false
/opt/splunkforwarder/etc/system/default/outputs.conf writeTimeout = 300
/opt/splunkforwarder/etc/system/local/outputs.conf [tcpout-server://10.46.249.41:9997]
/opt/splunkforwarder/etc/system/local/outputs.conf [tcpout:default-autolb-group]
/opt/splunkforwarder/etc/system/local/outputs.conf server = 10.22.139.99:9997

0 Karma
Reply

auraria1
Path Finder
‎07-19-2018 11:45 AM

When you created your outputs.conf where did you place the file?

Should be:

/directory to splunk/splunkforwarder/etc/system/local/outputs.conf

Be sure it says outputs.conf and not output.conf.

0 Karma
Reply

ahmemohs03
Explorer
‎07-19-2018 11:52 AM

Thanks .

location : /opt/splunkforwarder/etc/system/local

yes its outputs.conf

inside outputs.conf

root@psdlepkl4 local]# cat outputs.conf

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = 12.34.342.87:9997

[tcpout-server://12.34.342.87:9997

0 Karma
Reply

auraria1
Path Finder
‎07-19-2018 11:57 AM

as @patilsonali1729 please post the btool debug output.

Under [tcpout:default-autolb-group]
add:

disabled = 0

This shouldn't be an issue however.

Also add the close bracket ( ] )to
so this
[tcpout-server://12.34.342.87:9997

to this

[tcpout-server://12.34.342.87:9997]

^This is also not needed as you specified the server and the output in the default autolb group.

0 Karma
Reply

ahmemohs03
Explorer
‎07-19-2018 12:16 PM

Thanks for response.
Also add the close bracket ( ] )to already there may be my mistake

Do I need to add (disabled=0)?

0 Karma
Reply

auraria1
Path Finder
‎07-19-2018 12:20 PM

It doesn't hurt.

After that try starting the forwarder again and let me know.

0 Karma
Reply

patilsonali1729
Path Finder
‎07-19-2018 09:53 AM

can you paste the content of your outputs.conf here

0 Karma
Reply

ahmemohs03
Explorer
‎07-19-2018 10:02 AM

[root@psdlepkl4 local]# cat outputs.conf
[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = 12.34.342.87:9997

[tcpout-server://12.34.342.87:9997]

This on server where universal forwarder installed

0 Karma
Reply
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...