Getting Data In

Splunk Universal Forwarder only forwards one csv log

RecoMark0
Path Finder

Hello,
I am having an issue with the universal forwarder, where only one csv log gets sent to the index. We have multiple servers with the forwarder installed, and each server has the following in the inputs.conf:

#DocView logs
    [monitor://\\$COMPUTERNAME\s$\Logs\Audit\*.docView.csv]
    disabled = 0
    followTail = false
    sourcetype = Doc View
    crcSalt = <SOURCE>
    ignoreOlderThan = 2d

I have ran "splunk list monitor" from the bin folder, and csv files in that folder are listed correctly, however, only one shows up in splunk when I search: sourcetype="Doc View", and it is always the first one alphabetically. The files are named: "ProjectA.docview.csv", "ProjectB.docview.csv", "ProjectC.docview.csv". ProjectA will always be the only result in splunk.

I do not think it is a KBPS issue either, as I have not seen the warning in splunkd for a few weeks.

I do not think it is a security issue as the files are created the same exact way from our system.

Question:
Is there anything else I can check, or logs I can look at to see what the issue is? Has anyone seen this issue before?

Thank you!

Edit: I do not need this log to be read as a csv file. It can be treated as a normal log with the csv extension.

0 Karma
1 Solution

eabrown2
Engager

This appears to be an issue with the initCrc.

Without changing the length the csvs would not be picked up on rollover.

I tested this with adding to the inputs.conf:

initCrcLength=1024

All logs are grabbed now.

View solution in original post

eabrown2
Engager

This appears to be an issue with the initCrc.

Without changing the length the csvs would not be picked up on rollover.

I tested this with adding to the inputs.conf:

initCrcLength=1024

All logs are grabbed now.

RecoMark0
Path Finder

This seems to have done the trick, thank you. I tested it on one of the servers, and more than one csv is now coming through.

0 Karma

tom_frotscher
Builder

This looks like the csv's have a long header that is the same for all csv files. Then splunk reads the first bytes of the files and if they are the same, the file will not be processed, because splunk think it is the same file.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

Try adjusting your inputs for the following:

monitor://\\$COMPUTERNAME\s$\Logs\Audit\*.docView.csv

Change this to

monitor://\\$COMPUTERNAME\s$\Logs\Audit\*.docview.csv

I am not sure the case sensitivity is an issue on windows, however there might be something that its missing.

Also, I'd recommend keeping your sourcetypes as one word, without spaces. If you want to separate it, do something like

sourcetype=doc:view

0 Karma

tom_frotscher
Builder

Hi,

as a first step, take a look in the splunkd.log of the forwarders. Always a good starting point for investigations according to forwarders.

0 Karma

RecoMark0
Path Finder

the splunkd log is not really telling me anything, other than that the server is only trying to send one of the files, and not all of them. There are no warnings or errors in the log either 😞

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...