How to whitelist or blacklist log files in Zip Archives?

[bleinfelder]

Hi there,

I'm trying to monitor log files in zip archives, that contain additional data files, which I mustn't monitor.

How can a I specify whitelist/blacklist for files contained in a zip archive?

I tried using whitelist in inputs.conf stanza, but that whitelist causes splunk to ignore the zip file completely.

Any help appreciated.

Regards,

Bernd

[csaur_splunk]

You can create a transform that sends the events to the nullQueue from the your blacklist by matching the source value.
Example: This will ignore all events from a zip archive that are README.txt files
in transforms.conf example:

[archived_file_blacklist]
SOURCE_KEY = MetaData:Source
FORMAT = nullQueue
DEST_KEY = queue
REGEX = (README)\.txt$

in props.conf example:

[zip_archive_sourcetype]
TRANSFORMS = archived_file_blacklist

in inputs.conf example:

[monitor:///Users/myuser/zipfiles/*.zip]
sourcetype = zip_archive_sourcetype

[grijhwani]

Last time I saw this asked there was no inherent mechanism for it. The white/blacklist specification is for monitoring paths.

The only thing I can think of is to unpack the zips by script, and ingest files selectively.