Getting Data In

Filtering events from Docker

New Member

Hello,

we are successfully logging events from Docker hosts via Splunk adapter (HEC) into splunk.
Problems arise when we try to filter some unwanted events.
I tried to send those events to the NULL-Queue

props.conf

[host::<somehost>]
TRANSFORMS-nullQueueFilterSpecificEvents = filter_setNull_4_specific_events

transforms.conf

[filter_setNull_4_specific_events]
REGEX = <someregex>
DEST_KEY = queue
FORMAT = nullQueue

It doesn't work. Still getting all events unfiltered. One possible reason could be, that depending on the HEC endpoint the events may go through a "structured parsing queue" instead of the normal route and that transformations aren't possible.
Is this a possbiel explanation?
How can I detemine what endpoint is used?

Perhaps the log config in the docker-compose.yml can give some clarification:

logging:
  driver: splunk
  options:
    tag: "ct={{.Name}}"
    splunk-token: <sometoken>
    splunk-url: https://<splunkhost-fqdn>:<port>;
    splunk-format: raw
    splunk-insecureskipverify: "true"

The events are not json but unstructured.
Can someone give me a hint or did anyone already successfully filter docker events with transformations in props.conf/transforms.conf.
Thanks in advance!

Regards,
Jens

0 Karma
1 Solution

Communicator

Docker Splunk Logging driver sends events to the /event (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fevent) endpoint. And only /raw (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fraw) HEC endpoint supports indexing pipeline (see http://dev.splunk.com/view/event-collector/SP-CAAAE6P).
Event endpoint expects that you will control the delivery on your end. Unfortunately Splunk Logging Driver does not have support for /raw endpoint.

With the collectord (Monitoring Docker application https://splunkbase.splunk.com/app/3723/) we support filtering out the events base on the specific regexp pattern, see example https://www.outcoldsolutions.com/docs/monitoring-docker/v5/annotations/#example-2-dropping-messages

View solution in original post

0 Karma

Communicator

Docker Splunk Logging driver sends events to the /event (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fevent) endpoint. And only /raw (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fraw) HEC endpoint supports indexing pipeline (see http://dev.splunk.com/view/event-collector/SP-CAAAE6P).
Event endpoint expects that you will control the delivery on your end. Unfortunately Splunk Logging Driver does not have support for /raw endpoint.

With the collectord (Monitoring Docker application https://splunkbase.splunk.com/app/3723/) we support filtering out the events base on the specific regexp pattern, see example https://www.outcoldsolutions.com/docs/monitoring-docker/v5/annotations/#example-2-dropping-messages

View solution in original post

0 Karma

New Member

Just as I thought. Thank you outcoldman!

0 Karma