https://community.splunk.com/t5/Getting-Data-In/Forwarding-not-working-for-some-directories/m-p/74269#M15181 <P>I have a unix light forwarder that works fine for the normal default splunk forwarding (the scripts for CPU, ports, etc).</P> <P>So:<BR /> <CODE>[monitor:///var/log]<BR /> whitelist=(\.log|log$|messages$|mesg$|cron$|acpid$|\.out)<BR /> blacklist=(lastlog)<BR /> index=os<BR /> disabled = 0</CODE></P> <P>Produces the expected entries on my indexer.</P> <P>I added a statement for a custom directory (using a custom index):</P> <P><CODE>[monitor:///services/.../logs]<BR /> whitelist=(\.log$|log$|messages$|mesg$|cron$|acpid$|\.out$)<BR /> blacklist=(lastlog$|\.gz$|^splunk|\.xml$|\.conf|^metrics)<BR /> index=services<BR /> disabled = 0</CODE></P> <P>The index is created on the indexer, and 'splunk list monitor' shows the directories are being monitored on the forwarder (including many directories that shouldn't be, but I saw another question that answered that).</P> <P>With all that, nothing new is added to my new index. What needs to happen to see my data on my indexer?</P> Thu, 21 Oct 2010 01:19:04 GMT mbrunetto 2010-10-21T01:19:04Z https://community.splunk.com/t5/Getting-Data-In/Forwarding-not-working-for-some-directories/m-p/74269#M15181 <P>I have a unix light forwarder that works fine for the normal default splunk forwarding (the scripts for CPU, ports, etc).</P> <P>So:<BR /> <CODE>[monitor:///var/log]<BR /> whitelist=(\.log|log$|messages$|mesg$|cron$|acpid$|\.out)<BR /> blacklist=(lastlog)<BR /> index=os<BR /> disabled = 0</CODE></P> <P>Produces the expected entries on my indexer.</P> <P>I added a statement for a custom directory (using a custom index):</P> <P><CODE>[monitor:///services/.../logs]<BR /> whitelist=(\.log$|log$|messages$|mesg$|cron$|acpid$|\.out$)<BR /> blacklist=(lastlog$|\.gz$|^splunk|\.xml$|\.conf|^metrics)<BR /> index=services<BR /> disabled = 0</CODE></P> <P>The index is created on the indexer, and 'splunk list monitor' shows the directories are being monitored on the forwarder (including many directories that shouldn't be, but I saw another question that answered that).</P> <P>With all that, nothing new is added to my new index. What needs to happen to see my data on my indexer?</P> Thu, 21 Oct 2010 01:19:04 GMT https://community.splunk.com/t5/Getting-Data-In/Forwarding-not-working-for-some-directories/m-p/74269#M15181 mbrunetto 2010-10-21T01:19:04Z https://community.splunk.com/t5/Getting-Data-In/Forwarding-not-working-for-some-directories/m-p/74270#M15182 <P>If possible, use Firefox to visit the Splunk forwarder's management port: <A href="https://fwder:8089/services/admin/inputstatus/TailingProcessor:FileStatus" rel="nofollow">https://fwder:8089/services/admin/inputstatus/TailingProcessor:FileStatus</A></P> <P>And see what reason is provided for ignoring the files you wish to see indexed.</P> <P>If it isn't possible to use Firefox, consider: wget --user=foo --password=bar <A href="https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus" rel="nofollow">https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus</A> | xmllint --format -</P> <P>It's most likely just some collision with the regexes (using ... in a stanza creates an implicit whitelist). Let's see what the system thinks it's looking for.</P> Wed, 12 Jan 2011 03:35:41 GMT https://community.splunk.com/t5/Getting-Data-In/Forwarding-not-working-for-some-directories/m-p/74270#M15182 amrit 2011-01-12T03:35:41Z