Getting Data In

How to whitelist or blacklist log files in Zip Archives?

Path Finder

Hi there,

I'm trying to monitor log files in zip archives, that contain additional data files, which I mustn't monitor.

How can a I specify whitelist/blacklist for files contained in a zip archive?

I tried using whitelist in inputs.conf stanza, but that whitelist causes splunk to ignore the zip file completely.

Any help appreciated.

Regards,

Bernd

0 Karma

Splunk Employee
Splunk Employee

You can create a transform that sends the events to the nullQueue from the your blacklist by matching the source value.
Example: This will ignore all events from a zip archive that are README.txt files
in transforms.conf example:

[archived_file_blacklist]
SOURCE_KEY = MetaData:Source
FORMAT = nullQueue
DEST_KEY = queue
REGEX = (README)\.txt$

in props.conf example:

[zip_archive_sourcetype]
TRANSFORMS = archived_file_blacklist

in inputs.conf example:

[monitor:///Users/myuser/zipfiles/*.zip]
sourcetype = zip_archive_sourcetype

Motivator

Last time I saw this asked there was no inherent mechanism for it. The white/blacklist specification is for monitoring paths.

The only thing I can think of is to unpack the zips by script, and ingest files selectively.

0 Karma