Deployment Architecture

UF is not sending data to indexer

sivaranjiniG
Path Finder

Some one please help me here..

i am trying to monitor /var/log/audit/audit.log using universal forwarder and sending it to indexer.. but logs are not being sent to indexer..here is the log i m seeing in splunkd of forwarder 

 

08-06-2020 13:48:17.728 +0530 DEBUG TailingProcessor -   Item '/var/log/audit/audit.log' matches stanza: /var/log/audit/audit.log.
08-06-2020 13:48:17.728 +0530 DEBUG TailingProcessor -   Storing config '/var/log/audit/audit.log' for app ''.
08-06-2020 13:48:17.728 +0530 DEBUG TailingProcessor -   Entry is associated with 1 configuration(s).
08-06-2020 13:48:17.728 +0530 DEBUG TailReader -   Will attempt to read file: /var/log/audit/audit.log.
08-06-2020 13:48:17.730 +0530 DEBUG TailReader -   Got classified_sourcetype='linux_audit' and classified_charset='UTF-8'.
08-06-2020 13:48:17.730 +0530 DEBUG WatchedFile - Storing pending metadata for file=/var/log/audit/audit.log, sourcetype=linux_audit, charset=UTF-8
08-06-2020 13:48:17.730 +0530 DEBUG WatchedFile - setting trailing nulls to false via 'true' or 'false' from conf'
08-06-2020 13:48:17.730 +0530 DEBUG WatchedFile -   Loading state from fishbucket.
08-06-2020 13:48:17.731 +0530 DEBUG WatchedFile -   Attempting to load indexed extractions config from conf=source::/var/log/audit/audit.log|host::xxx|linux_audit|3 ...
08-06-2020 13:48:17.731 +0530 DEBUG WatchedFile -   Reading for plain initCrc...
08-06-2020 13:48:17.731 +0530 DEBUG WatchedFile -   initcrc has changed to: 0x153ce0cdaa107eee.
08-06-2020 13:48:17.731 +0530 DEBUG WatchedFile - Record found, will advance file by offset=12920 initcrc=0x153ce0cdaa107eee.
08-06-2020 13:48:17.731 +0530 DEBUG WatchedFile -   Creating new pipeline input channel with channel id: 4.
08-06-2020 13:48:17.732 +0530 DEBUG WatchedFile -   Attempting to load indexed extractions config from conf=source::/var/log/audit/audit.log|host::xxx|linux_audit|4 ...
08-06-2020 13:48:17.732 +0530 DEBUG TailReader - About to read data (Opening file: /var/log/audit/audit.log).
08-06-2020 13:48:17.732 +0530 DEBUG WatchedFile - seeking /var/log/audit/audit.log to off=12920
08-06-2020 13:48:17.732 +0530 DEBUG WatchedFile - Reached EOF: /var/log/audit/audit.log (read 0 bytes)
08-06-2020 13:48:17.732 +0530 DEBUG TailReader - Hit EOF immediately.
08-06-2020 13:48:17.732 +0530 DEBUG TailReader - Have definitely hit EOF.
08-06-2020 13:48:17.732 +0530 DEBUG TailReader - Finished reading file='/var/log/audit/audit.log' in tailreader0 thread, disposition=ACKNOWLEDGE_CHANGE, deferredBy=0.000
08-06-2020 13:48:17.732 +0530 DEBUG TailReader - Returning disposition=ACKNOWLEDGE_CHANGE for file=/var/log/audit/audit.log
08-06-2020 13:48:17.732 +0530 DEBUG TailReader - Start reading file="/opt/splunkforwarder/var/log/splunk/splunkd.log" in tailreader0 thread

 

Labels (2)
0 Karma
1 Solution

sivaranjiniG
Path Finder

Thanks For all the Answers...But after deleting fishbucket, data are being pushed to indexer

View solution in original post

0 Karma

thambisetty
SplunkTrust
SplunkTrust

is Splunk universal forwarder running with root account?

if not, switch user to account which is used by splunk service and try to read file "/var/log/audit/audit.log" using cat or tail or more.

let me know if you are seeing results or not. if not how did you set permission to read file by splunk user?

 

 

 

————————————
If this helps, give a like below.
0 Karma

thambisetty
SplunkTrust
SplunkTrust

can you confirm if any other files are monitored on the server being sent to Indexer?

————————————
If this helps, give a like below.
0 Karma

sivaranjiniG
Path Finder

yes 

/var/log/messages
/var/log/rhsm
/var/log/secure
/var/log/yum.log
/var/log/cron
/var/log/maillog

these are being sent to indexer without any issue

0 Karma

aashiqwork
Explorer

Go to the Search Head and search with the below search (Make sure you have rights to see internal indexes data):

index=_internal | dedup host | fields host | table host

Look in the list to see if your Forwarder’s hostname is in the list, if it is present that means the Forwarder is connected. 

0 Karma

sivaranjiniG
Path Finder

Thanks For all the Answers...But after deleting fishbucket, data are being pushed to indexer

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sivaranjiniG,

this means that you're indexing twice logs.

Ciao and next time.

Giuseppe

0 Karma

sivaranjiniG
Path Finder

I never have got data in indexer before deleting fishbucket so i thought it didnt read log file..moreover i didnt see any crc related issue in splunkd log..i am not sure how to find out these types of issue in splunkd log

0 Karma

thambisetty
SplunkTrust
SplunkTrust

check below answer may help you.

 

https://community.splunk.com/t5/Getting-Data-In/eof-error-when-reading-file/td-p/52833

————————————
If this helps, give a like below.
0 Karma

sivaranjiniG
Path Finder

forwarder is not running with root account but Splunk user has access to read auditd log..that should be enough to read logs..i guess


0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...