Deployment Architecture

Why am I not receiving any Data from the Linux Server?

JarrettM
Path Finder

No data being received from Linux client.

Running Splunk Enterprise 7.0.3 on Windows Server 2012 R2. Receiving data across 7 indexes from 36 Windows Universal Forwarder clients and two syslog servers. Installed Splunk Universal Forwarder 7.0.3 on Redhat 7 server. Client phones home and makes good SSL connection:

5/10/18 8:53:54.613 AM 10..x.x.x - - [10/May/2018:08:53:54.613 -0400] "POST /services/broker/phonehome/connection_10..x.x.x_8089_ SPLUNK_CLIENT _ SPLUNK_CLIENT _784C7242-6306-403F-B877-11B04B59FFC7 HTTP/1.1" 200 832 - - - 0ms

05-10-2018 08:53:49.446 -0400 INFO  Metrics - group=tcpin_connections, 10.x.x.x:48460:9997, connectionType=cookedSSL, sourcePort=48460, sourceHost=10.x.x.x, sourceIp=10.x.x.x, destPort=9997, kb=72.01, _tcp_Bps=2378.52, _tcp_KBps=2.32, _tcp_avg_thruput=2.77, _tcp_Kprocessed=3107.70, _tcp_eps=3.55, _process_time_ms=0, evt_misc_kBps=0.06, evt_raw_kBps=1.03, evt_fields_kBps=1.13, evt_fn_kBps=0.84, evt_fv_kBps=0.29, evt_fn_str_kBps=0.06, evt_fn_meta_dyn_kBps=0.00, evt_fn_meta_predef_kBps=0.03, evt_fn_meta_str_kBps=0.68, evt_fv_num_kBps=0.03, evt_fv_str_kBps=0.23, evt_fv_predef_kBps=0.00, evt_fv_offlen_kBps=0.00, evt_fv_fp_kBps=0.00, build=fa31da744b51, version=7.0.3, os=Linux, arch=x86_64, hostname=SPLUNK_CLIENT, guid=784C7242-6306-403F-B877-11B04B59FFC7, fwdType=full, ssl=true, lastIndexer=y.y.y.y:9997, ack=false destPort =9997 host =SPLUNK_SERVER     support-server hostname =SPLUNK_CLIENT log_level =INFO source =E:\Splunk\var\log\splunk\metrics.log sourceHost =10.x.x.x sourcePort =   48460 tag =support-server

Deployment App shows in Client's Apps directory:
#var-logs
[monitor:///var/log/*.log]
disabled=false
index=linux
sourcetype=var-logs

Index and sourcetype are correct. 

These are the only lines the the Clients splunkd that indicate any sort of problem:

05-10-2018 09:04:03.532 -0400 WARN  LineBreakingProcessor - Truncating line because limit of 10000 bytes has been exceeded with a line length >= 12323 - data_source="/home/USER_NAME/splunk/var/log/splunk/.splunkd.log.swp", data_host=" SPLUNK_CLIENT ", data_sourcetype="swp-too_small"

05-10-2018 09:04:03.533 -0400 WARN  TailReader - Insufficient permissions to read file='/home/USER_NAME/splunk/var/log/splunk/.splunkd.log.swx' (hint: No such file or directory ,                            UID: 474401269, GID: 474400513).

05-10-2018 09:04:06.536 -0400 WARN  FileClassifierManager - The file '/home/USER_NAME/splunk/var/log/splunk/.splunkd.log.swp' is invalid. Reason: binary

Any ideas?

Thanks

0 Karma

JarrettM
Path Finder

Succeeded in getting data by using the the following command on the client:

./splunk add monitor /var/log

however it still will not work by using a deployed app from the deployment server.

0 Karma

JarrettM
Path Finder

Never Mind!

Here was the problem:

Default instead of default

Doh!

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...