topic Re: Can't replace a healthcheck string in nginx in Splunk Search

Can't replace a healthcheck string in nginx

scalp42 — Wed, 31 Oct 2012 00:01:40 GMT

Hi,

I have looked at the docs and tried to remove a line from nginx access log regarding our LB :

192.168.27.169 - - [30/Oct/2012:23:02:53 +0000] "GET /node/lbtest.txt HTTP/1.0" 200 9 "-" "HTTP-Monitor/1.1" "-"

and

Started GET "/node/lbtest.txt" for 127.0.0.1 at 2012-10-30 23:55:58 +0000
Processing by HealthCheckController#lbtest as TXT

Here is my props.conf :

[sourcetype::access_combined_wcookie]
TRANSFORMS-ignore=ignore

[sourcetype::production-2]
TRANSFORMS-null=setnull

[sourcetype::access_combined_wcookie]
TRANSFORMS-null2=nukefromorbit

[host::app*]
SEDCMD-health = s/lbtest/DEVOPS/g

Please note that production-2, access_combined_wcookie sourcetypes parse Nginx logs.

The host sending the event is app-05.

Here is my transforms.conf :

[ignore]
REGEX = (?m)*lbtest*
DEST_KEY = queue
FORMAT = nullQueue

[setnull]
REGEX = lbtest|HealthCheckController
DEST_KEY = queue
FORMAT = nullQueue

[nukefromorbit]
REGEX = *
DEST_KEY = queue
FORMAT = nullQueue

This conf is obviously destructive by nature (as in, way beyond removing this lbtest line, mix-n-matching), as I've tried anything possible to remove this line from the logs.

I have restarted splunk forwarder and I'm running out of solutions.

Thank you in advance.

Re: Can't replace a healthcheck string in nginx

gkanapathy — Wed, 31 Oct 2012 06:14:16 GMT

Are you sure this configuration is in the right place? See http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Configurationparametersandthedatapipeline

Re: Can't replace a healthcheck string in nginx

scalp42 — Wed, 31 Oct 2012 18:18:47 GMT

I'm pretty sure it is :

Parsing

props.conf

LINE_BREAKER, SHOULD_LINEMERGE, BREAK_ONLY_BEFORE_DATE, and all other line     merging settings
TZ, DATETIME_CONFIG, TIME_FORMAT, TIME_PREFIX, and all other time extraction     settings and rules
TRANSFORMS* which includes per-event queue filtering, per-event index    assignment, per-event routing. Applied in the order defined
SEDCMD*
MORE_THAN*, LESS_THAN*

transforms.conf`

stanzas referenced by a TRANSFORMS* clause in props.conf

Re: Can't replace a healthcheck string in nginx

gkanapathy — Wed, 31 Oct 2012 18:53:03 GMT

I guess my point was, is it on the right server?

Re: Can't replace a healthcheck string in nginx

scalp42 — Wed, 31 Oct 2012 23:13:15 GMT

I think it has to be on the forwarder/nginx host.