How to edit props.conf to collect gz.done files from Blue Coat's proxy FTP server?

[daniel_augustyn]

How to edit props.conf to start collecting gz.done files from Blue Coat's proxy FTP server? Reporter change .gz files to gz.done files. What should I do to start pushing these files via universal forwarder to the indexers.

[daniel_augustyn · Accepted solution]

I can't find gzip2 file in the bin folder.

[MuS]

Sorry, my Windows not-knowledge got me here. There is no bzip2 shipped with the Windows UF.
I found some powershell command which could do it, but that looks complicated http://stackoverflow.com/questions/17546016/how-can-you-zip-or-unzip-from-the-command-prompt-using-o... other option would be install gzip2 or bzip2 on the UF and use the unarchive_cmd= gzip -d or unarchive_cmd= bzip -d in props.conf

Sorry if this does not answer your question or is helpful.....

[MuS]

Hi daniel_augustyn,

on your universal forwarder, check the inputs.conf currently monitoring the path holding the .gz files. Check if there is a whitelist= or a blacklist for this stanza and modify it according to your needs.
See the docs on whitelist or blacklist http://docs.splunk.com/Documentation/Splunk/6.3.1/Admin/Inputsconf

Hope this helps ...

cheers, MuS

[daniel_augustyn]

How can I start collecting "gz.done" files?

[MuS]

check the inputs.conf and verify if those files are blacklisted or not. Also check if there is a whitelist; if so add them to the whitelist regex and they will be monitored (Some times you need to restart the universal forwarder)

[daniel_augustyn]

That's what I have:

[monitor://E:\Server1\BCT-GW-SG\*.done]
sourcetype = bluecoat:proxysg:access:file
disabled = false
index=proxy

[daniel_augustyn]

And it doesn't collect these files.

[MuS]

Is the forwarder process able to read those files? permission issue? any errors related to this monitor in splunkd.log?

[daniel_augustyn]

I am just fine with reading .gz files, I can't read gz.done files from the same folder.

[daniel_augustyn]

no errors, would this be something related to https://answers.splunk.com/answers/8521/can-splunk-read-bluecoat-logs-formatted-and-compressed-as-lo...

[MuS]

My bad sorry thought this was no longer needed.....yes, try this option unarchive_cmd= in props.conf to tell Splunk how to handle the gz.done file

[daniel_augustyn]

would that work on the Windows box?

[MuS]

Well you should find bzip2 in the Splunk bin directory so you should be able to run it.

[MuS]

Okay, I must admit my not-knowledge of Windows got me here 🙂
The universal forwarder on Windows does not come with bzip2 and therefore you cannot just use the unarchive_cmd = bzip2 -d option.
I found some powershell command which could do such a thing, but it looks complicated http://stackoverflow.com/questions/17546016/how-can-you-zip-or-unzip-from-the-command-prompt-using-o...
Other option, install gzip or zip on this forwarder and use it in the unarchive_cmd option.

[daniel_augustyn]

I can't find bzip2 in the bin directory, is there a way to threat done like gz files.

[daniel_augustyn]

Would you mind sharing stanza for it?

[daniel_augustyn]

Can you let me know what the stanza should be?