Deployment Architecture

Universal Forwarder with logrotate - events only arrive once a week

mprilop
Explorer

Hi,
i am having problems troubleshooting missing log entries. The UF seems to be configured correctly (list monitor and splunkd.log indicate everything okay). But only once a week four events reach splunk (it should be several thousand per day). Once per week may point to logrotate which is done weekly (config below) but i could not find any reference to similar problems with current versions of UF and splunk.

Our configuration:
Splunk version 6.1.3 (some enterprise license)

Universal Forwarder

# cat /opt/splunkforwarder/etc/splunk.version 
VERSION=6.1.3
BUILD=220630
PRODUCT=splunk
PLATFORM=Linux-x86_64

monitoring entry:

[monitor:///var/www/*/shared/log/*.log]
disabled = false
sourcetype = rails_app

splunkd.log entry for above monitoring:

01-14-2015 10:36:23.714 +0100 INFO  WatchedFile - Will begin reading at offset=72355036 for file='/var/www/our_app/shared/log/production.log'.

logrotate setting:

/var/www/*/shared/log/*.log {
    weekly
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    copytruncate
}
0 Karma
1 Solution

mprilop
Explorer

Turns out i had insufficient rights to view the resulting logs.

I do not know why i still saw single entries and combined with me being able to see all other logs directed to splunk via syslog it never occurred to me to check my access rights with our hoster.

View solution in original post

0 Karma

mprilop
Explorer

Turns out i had insufficient rights to view the resulting logs.

I do not know why i still saw single entries and combined with me being able to see all other logs directed to splunk via syslog it never occurred to me to check my access rights with our hoster.

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi mprilop,

take a look at this answer http://answers.splunk.com/answers/185453/why-copytruncate-logrotate-does-not-play-well-with.html provided by @yannK, it includes some work around as well.

cheers, MuS

mprilop
Explorer

Thanks for your suggestion. I will try out the workaround and hope it also solves my problem.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...