How to reconfigure syslog-ng when file is getting updated but data isn't getting into indexer (Universal Forwarder)?

[martinnepolean]

Hi,

We have configured syslog-ng to send data to indexers, Sometimes, the syslog file is getting updated but data is not getting into indexer. For the same index we are getting data from other files but only one file is not getting indexed. We have to restart syslog splunk process to get the indexing

We have below setting configured already.

[thruput]
maxKBps = 0

[inputproc]
max_fd = 1000

[tcpout:targetgrpmaster]
autoLBFrequency = 30
indexerDiscovery = masterindexdiscovery
forceTimebasedAutoLB = True
maxQueueSize = 128MB

[general]
parallelIngestionPipelines = 6

[queue]
maxSize = 256MB

[queue=parsingQueue]
maxSize = 128MB

[isoutamo]

Hi

can you explain this little bit more?

Are you running syslog-ng on the same box and collecting events to file(s) by it and then ingesting those with splunk or are you sending events directly to splunk udp/tcp port?

Can you share your input definitions?

r. Ismo

[martinnepolean]

Hi Ismo,

Yes, we are using syslog-ng and collect data using UCP&TCP to log file and using UF inputs.conf and outputs.conf, we are sending it to splunk indexers

[monitor:///opt/syslog_ng/logs/juniper/.../*.log]
sourcetype = juniper:firewall
index = juniper_index
#_TCP_ROUTING = splunkprodhf
ignoreOlderThan = 30d
disabled = false
host_segment = 5

[isoutamo]

Hi

this inputs.conf seems to be used for several nodes. Are all other working or have those also issues time by time? 

Are there any other inputs or is these syslog files only in use?

The node which you are using is linux UF with splunk UF version x.x.x?

r. Ismo