I have a source log that sometimes contains binary characters. Splunk is not indexing any events for this source type. The source type in question was configured as follow at the universal forwarder:
This configuration should work but it is not... Any idea...
I tried with NO_BINARY_CHECK = true and NO_BINARY_CHECK = false.
Thanks,
Lp
inputs.conf:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
_blacklist=\.(gz|log.*|out.*|run.properties)$
disabled = false
props.conf:
[azkaban]
NO_BINARY_CHECK = true
The problem is that this source type is not being indexed and events are being appended. Splunkd.log does not complain about it.
12-11-2012 14:24:37.318 +0000 INFO TailingProcessor - Parsing configuration stanza: monitor:///usr/local/azkaban/logs/azkaban.log.
Btool reports:
inputs
[monitor:///usr/local/azkaban/logs/azkaban.log]
_blacklist = \.(gz|log.*|out.*|run.properties)$
_rcvbuf = 1572864
blacklist = \.(gz|log.*|out.*)$
disabled = false
host = abc.com
index = default
sourcetype = azkaban
props
[azkaban]
ANNOTATE_PUNCT = True
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
NO_BINARY_CHECK = true
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
TZ = UTC
My original config was this one:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
_blacklist=\.(gz|log.*|out.*|run.properties)$
disabled = false
I changed it to:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
blacklist=.(gz|log.*|out.*|run.properties)$
disabled = false
And the source type is back to work.
Thanks,
Lp
My original config was this one:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
_blacklist=\.(gz|log.*|out.*|run.properties)$
disabled = false
I changed it to:
[monitor:///usr/local/azkaban/logs/azkaban.log]
sourcetype = azkaban
blacklist=.(gz|log.*|out.*|run.properties)$
disabled = false
And the source type is back to work.
Thanks,
Lp
[monitor:///usr/local/azkaban/logs/azkaban.log]
_blacklist = .(gz|log.|out.|run.properties)$
_rcvbuf = 1572864
blacklist = .(gz|log.|out.)$
disabled = false
host = abc.com
index = default
sourcetype = azkaban
You are blacklisting .log
For the benefit of doubt I tried your recommended blacklist. It worked. I do not see why the _blacklist presented in original question should not work.
Thanks,
LP
I disagree. I am blacklisting ".log.". See the initial question.
_blacklist=.(gz|log.|out.|run.properties)$
Your config looks wrong to me, from the docs;
NO_BINARY_CHECK = [true|false]
* When set to true, Splunk processes binary files.
* Can only be used on the basis of [<sourcetype>], or [source::<source>], not [host::<host>].
* Defaults to false (binary files are ignored).
At the moment you have it configured to ignore binary files
I tried both ways:
NO_BINARY_CHECK = true and NO_BINARY_CHECK = false
Still does not work. Any ideas...