Getting Data In

How to reconfigure syslog-ng when file is getting updated but data isn't getting into indexer (Universal Forwarder)?

martinnepolean
Explorer

Hi,

We have configured syslog-ng to send data to indexers, Sometimes, the syslog file is getting updated but data is not getting into indexer. For the same index we are getting data from other files but only one file is not getting indexed. We have to restart syslog splunk process to get the indexing

We have below setting configured already.

[thruput]
maxKBps = 0

[inputproc]
max_fd = 1000

[tcpout:targetgrpmaster]
autoLBFrequency = 30
indexerDiscovery = masterindexdiscovery
forceTimebasedAutoLB = True
maxQueueSize = 128MB

[general]
parallelIngestionPipelines = 6

[queue]
maxSize = 256MB

[queue=parsingQueue]
maxSize = 128MB

 

Labels (3)
Tags (1)
0 Karma

soutamo
SplunkTrust
SplunkTrust

Hi

can you explain this little bit more?

Are you running syslog-ng on the same box and collecting events to file(s) by it and then ingesting those with splunk or are you sending events directly to splunk udp/tcp port?

Can you share your input definitions?

r. Ismo

0 Karma

martinnepolean
Explorer

Hi Ismo,

 

Yes, we are using syslog-ng and collect data using UCP&TCP to log file and using UF inputs.conf and outputs.conf, we are sending it to splunk indexers

 

 

[monitor:///opt/syslog_ng/logs/juniper/.../*.log]
sourcetype = juniper:firewall
index = juniper_index
#_TCP_ROUTING = splunkprodhf
ignoreOlderThan = 30d
disabled = false
host_segment = 5

0 Karma

soutamo
SplunkTrust
SplunkTrust

Hi

this inputs.conf seems to be used for several nodes. Are all other working or have those also issues time by time? 

Are there any other inputs or is these syslog files only in use?

The node which you are using is linux UF with splunk UF version x.x.x?

r. Ismo

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!