Getting Data In

Filtering events from Docker

JenWun
New Member

Hello,

we are successfully logging events from Docker hosts via Splunk adapter (HEC) into splunk.
Problems arise when we try to filter some unwanted events.
I tried to send those events to the NULL-Queue

props.conf

[host::<somehost>]
TRANSFORMS-nullQueueFilterSpecificEvents = filter_setNull_4_specific_events

transforms.conf

[filter_setNull_4_specific_events]
REGEX = <someregex>
DEST_KEY = queue
FORMAT = nullQueue

It doesn't work. Still getting all events unfiltered. One possible reason could be, that depending on the HEC endpoint the events may go through a "structured parsing queue" instead of the normal route and that transformations aren't possible.
Is this a possbiel explanation?
How can I detemine what endpoint is used?

Perhaps the log config in the docker-compose.yml can give some clarification:

logging:
  driver: splunk
  options:
    tag: "ct={{.Name}}"
    splunk-token: <sometoken>
    splunk-url: https://<splunkhost-fqdn>:<port>;
    splunk-format: raw
    splunk-insecureskipverify: "true"

The events are not json but unstructured.
Can someone give me a hint or did anyone already successfully filter docker events with transformations in props.conf/transforms.conf.
Thanks in advance!

Regards,
Jens

0 Karma
1 Solution

outcoldman
Communicator

Docker Splunk Logging driver sends events to the /event (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fevent) endpoint. And only /raw (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fraw) HEC endpoint supports indexing pipeline (see http://dev.splunk.com/view/event-collector/SP-CAAAE6P).
Event endpoint expects that you will control the delivery on your end. Unfortunately Splunk Logging Driver does not have support for /raw endpoint.

With the collectord (Monitoring Docker application https://splunkbase.splunk.com/app/3723/) we support filtering out the events base on the specific regexp pattern, see example https://www.outcoldsolutions.com/docs/monitoring-docker/v5/annotations/#example-2-dropping-messages

View solution in original post

0 Karma

outcoldman
Communicator

Docker Splunk Logging driver sends events to the /event (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fevent) endpoint. And only /raw (https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fraw) HEC endpoint supports indexing pipeline (see http://dev.splunk.com/view/event-collector/SP-CAAAE6P).
Event endpoint expects that you will control the delivery on your end. Unfortunately Splunk Logging Driver does not have support for /raw endpoint.

With the collectord (Monitoring Docker application https://splunkbase.splunk.com/app/3723/) we support filtering out the events base on the specific regexp pattern, see example https://www.outcoldsolutions.com/docs/monitoring-docker/v5/annotations/#example-2-dropping-messages

0 Karma

JenWun
New Member

Just as I thought. Thank you outcoldman!

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...