Getting Data In
Highlighted

How to send some syslog messages to nullQueue - naive config not working

New Member

Running Enterprise 8.0.2.1. Data is coming in from a universal forwarder with index=syslog sourcetype=syslog and I'm trying to filter out unwanted messages. Here's a sample of the data:

2020-04-05T20:06:41.435487+00:00 HOST123 2020-04-05 20:06:41,424 Level="INFO" Name="support.bfcp" Message="Received BFCP message" Dst-address="x.x.x.x" Dst-port="41890" Src-address="y.y.y.y" Src-port="28888" Call-id="00000000-1111-2222-3333-444444444444" Primitive="Hello" Transaction-id="1014"
2020-04-05T20:06:37.552312+00:00 HOST123 2020-04-05 20:06:37,551 Level="INFO" Name="support.ice" Message="ICE new-local-candidate event" Media-type="h224" Stream-id="4" Component-id="RTCP" Local-candidate-type="host" Local-candidate-address="x.x.x.x" Local-candidate-port="41659" Local-candidate-transport="udp" Call-id="None"
2020-04-05T20:09:08.286431+00:00 HOST123 2020-04-05 20:09:08,269 Level="INFO" Name="support.participant" Message="Media Stream created" Participant="Patient" Call-id="00000000-1111-2222-3333-444444444444" Conversation-id="00000000-1111-2222-3333-444444444444" Detail="Stream 1 (video)"

I want to send certain events to nullQueue based on the Name="blah" field, so I naively did the following on the indexer:

/opt/splunk/etc/system/local/props.conf:

[syslog]
TRANSFORMS-mysystem = mysystem-nullqueue

/opt/splunk/etc/system/local/transforms.conf:

[mysystem-nullqueue]
DEST_KEY = queue
REGEX = Name=\"support\.(ice|bfcp|sip|rest|h323|dns)
FORMAT = nullQueue

Output of splunk cmd btool XXX list --debug for XXX=transforms/props:

/opt/splunk/etc/system/local/transforms.conf                           [mysystem-nullqueue]
/opt/splunk/etc/system/default/transforms.conf                         CAN_OPTIMIZE = True
/opt/splunk/etc/system/default/transforms.conf                         CLEAN_KEYS = True
/opt/splunk/etc/system/default/transforms.conf                         DEFAULT_VALUE =
/opt/splunk/etc/system/default/transforms.conf                         DEPTH_LIMIT = 1000
/opt/splunk/etc/system/local/transforms.conf                           DEST_KEY = queue
/opt/splunk/etc/system/local/transforms.conf                           FORMAT = nullQueue
/opt/splunk/etc/system/default/transforms.conf                         KEEP_EMPTY_VALS = False
/opt/splunk/etc/system/default/transforms.conf                         LOOKAHEAD = 4096
/opt/splunk/etc/system/default/transforms.conf                         MATCH_LIMIT = 100000
/opt/splunk/etc/system/default/transforms.conf                         MV_ADD = False
/opt/splunk/etc/system/local/transforms.conf                           REGEX = Name=\"support\.(ice|bfcp|sip|rest|h323|dns)
/opt/splunk/etc/system/default/transforms.conf                         SOURCE_KEY = _raw
/opt/splunk/etc/system/default/transforms.conf                         WRITE_META = False

/opt/splunk/etc/apps/search/local/props.conf                      [syslog]
/opt/splunk/etc/system/default/props.conf                         ADD_EXTRA_TIME_FIELDS = True
/opt/splunk/etc/system/default/props.conf                         ANNOTATE_PUNCT = True
/opt/splunk/etc/system/default/props.conf                         AUTO_KV_JSON = true
/opt/splunk/etc/system/default/props.conf                         BREAK_ONLY_BEFORE =
/opt/splunk/etc/system/default/props.conf                         BREAK_ONLY_BEFORE_DATE = True
/opt/splunk/etc/system/default/props.conf                         CHARSET = UTF-8
/opt/splunk/etc/system/default/props.conf                         DATETIME_CONFIG = /etc/datetime.xml
/opt/splunk/etc/system/default/props.conf                         DEPTH_LIMIT = 1000
/opt/splunk/etc/apps/search/local/props.conf                      EXTRACT-mysystem-syslog-apache = apache2.\d+.: (?<srcip>\S+).*?\"(?<method>\S+) (?<url>[^ ?]+)\?*(?<query>\S*) \S+\" \S+ (?<respcode>\d+) (?<respbytes>\S+) (?<resptime>\d+)
/opt/splunk/etc/apps/search/local/props.conf                      FIELDALIAS-syslog_dst_address = Dst_address ASNEW dest Dst_port ASNEW dest_port Src_address ASNEW src Src_port ASNEW src_port
/opt/splunk/etc/system/default/props.conf                         HEADER_MODE =
/opt/splunk/etc/system/default/props.conf                         LEARN_MODEL = true
/opt/splunk/etc/system/default/props.conf                         LEARN_SOURCETYPE = true
/opt/splunk/etc/system/default/props.conf                         LINE_BREAKER_LOOKBEHIND = 100
/opt/splunk/etc/system/default/props.conf                         MATCH_LIMIT = 100000
/opt/splunk/etc/system/default/props.conf                         MAX_DAYS_AGO = 2000
/opt/splunk/etc/system/default/props.conf                         MAX_DAYS_HENCE = 2
/opt/splunk/etc/system/default/props.conf                         MAX_DIFF_SECS_AGO = 3600
/opt/splunk/etc/system/default/props.conf                         MAX_DIFF_SECS_HENCE = 604800
/opt/splunk/etc/system/default/props.conf                         MAX_EVENTS = 256
/opt/splunk/etc/system/default/props.conf                         MAX_TIMESTAMP_LOOKAHEAD = 32
/opt/splunk/etc/system/default/props.conf                         MUST_BREAK_AFTER =
/opt/splunk/etc/system/default/props.conf                         MUST_NOT_BREAK_AFTER =
/opt/splunk/etc/system/default/props.conf                         MUST_NOT_BREAK_BEFORE =
/opt/splunk/etc/system/default/props.conf                         REPORT-syslog = syslog-extractions
/opt/splunk/etc/system/default/props.conf                         SEGMENTATION = indexing
/opt/splunk/etc/system/default/props.conf                         SEGMENTATION-all = full
/opt/splunk/etc/system/default/props.conf                         SEGMENTATION-inner = inner
/opt/splunk/etc/system/default/props.conf                         SEGMENTATION-outer = outer
/opt/splunk/etc/system/default/props.conf                         SEGMENTATION-raw = none
/opt/splunk/etc/system/default/props.conf                         SEGMENTATION-standard = standard
/opt/splunk/etc/system/default/props.conf                         SHOULD_LINEMERGE = False
/opt/splunk/etc/system/default/props.conf                         TIME_FORMAT = %b %d %H:%M:%S
/opt/splunk/etc/system/default/props.conf                         TRANSFORMS = syslog-host
/opt/splunk/etc/system/local/props.conf                           TRANSFORMS-mysystem = mysystem-nullqueue
/opt/splunk/etc/system/default/props.conf                         TRUNCATE = 10000
/opt/splunk/etc/system/default/props.conf                         category = Operating System
/opt/splunk/etc/system/default/props.conf                         description = Output produced by many syslog daemons, as described in RFC3164 by the IETF
/opt/splunk/etc/system/default/props.conf                         detect_trailing_nulls = false
/opt/splunk/etc/system/default/props.conf                         maxDist = 3
/opt/splunk/etc/system/default/props.conf                         priority =
/opt/splunk/etc/system/default/props.conf                         pulldown_type = true
/opt/splunk/etc/system/default/props.conf                         sourcetype =

After a config refresh or a restart of Splunk, the syslog index is still adding new entries containing Name="support.rest" or Name="support.ice". How do I further debug nullQueue not working?

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

Ultra Champion

your btool output:
/opt/splunk/etc/apps/search/local/props.conf [syslog]
This is not /opt/splunk/etc/system/local/props.conf:
but
/opt/splunk/etc/system/local/props.conf TRANSFORMS-mysystem = mysystem-nullqueue
something is wrong.

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

New Member

Yeah I'm assuming that etc/apps/search/local is there because a field lookup was added for sourcetype=syslog in the Splunk UI, referenced by EXTRACT-mysystem-syslog-apache and FIELDALIAS-syslogdstaddress lines.

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

Ultra Champion
0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

New Member

Doesn't that precedence only apply to configuration items that appear in multiple locations, and thus need to be overridden in some defined order?

That is, even if app config is higher priority here, there's no TRANSFORMS or TRANSFORMS-* items to process app-wise, thus we get the debug output showing that the active config items are TRANSFORMS = syslog-host and TRANSFORMS-mysystem = mysystem-nullqueue.

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

Ultra Champion

Why not do you write props.conf with nullqueue under etc/apps?

Doesn't that precedence only apply to configuration items that appear in multiple locations, and thus need to be overridden in some defined order?
I don't know how it works. I hope someone answers.

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

New Member

Restarting after moving/duplicating the settings into etc/apps/search/local/ files still doesn't filter out any of the syslog data.

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

Ultra Champion

how 's btool output?
new syslog message doesn't filter out?

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

New Member

The btool output is the same as before, but the "TRANSFORMS-mysystem = mysystem-nullqueue" line is showing as coming from the search files instead of the etc/system/local files, same with the transforms.conf header/dest_key/format/regex lines. All the unwanted syslog messages are still making it through to the index.

0 Karma
Highlighted

Re: How to send some syslog messages to nullQueue - naive config not working

Ultra Champion

btool output:

 /opt/splunk/etc/apps/search/local/props.conf  [syslog]
 /opt/splunk/etc/apps/search/local/props.conf  TRANSFORMS-mysystem = mysystem-nullqueue 

These have to be this.

0 Karma