Unable to delete some logs with TRANSFORMS

[azanoli]

Hi all

I have problems with 2 transforms for a Cisco IronPort Proxy. I receive 2 different types of logs:

Pacfile download:

Feb 3 15:08:22 ironport2.bank.com Feb 03 15:08:22 pacfile-splunk: Info: 189.149.11.14 - /proxy-ntlm.pac is downloaded successfully

Proxy-Access:

Jan 8 09:22:19 proxy.bank.com Jan 08 09:22:19 accesslogs-splunk: Info: 1420705339.401 307 189.149.128.70 TCP_CLIENT_REFRESH_MISS/200 6439 CONNECT tunnel://eqi.ibb.ubs.com:443/ "MAIN\bsm@MAIN.NTLM" DIRECT/eqi.ibb.ubs.com - PASSTHRU_CUSTOMCAT_7-DefaultGroup-ID.G.URL.Domains.Whitelist-DefaultGroup-NONE-NONE-DefaultGroup <C_G.N0,-,-,"-",-,-,-,-,"-",-,-,-,"-",-,-,"-","-",-,-,-,-,"-","-","-","-","-","-",167.79,0,-,"-","-",-,"-",-,-,"-","-"> - "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" - 216.219.78.86 - "G.NoDecryption.BypassScanning" 3438

I want to remove the Pacfile entries with the transform cisco_wsa_pacfile_drop. After them I have to reformat the Proxy-Access logs to delete all entries until the timestamp with the transform cisco_wsa_format_clean.
The second part works well. But the deletion of the Pacfile logs doesn't work.

--- props.conf ---
[cisco:wsa:squid]
TRANSFORMS-wsa_format = cisco_wsa_format_clean,cisco_wsa_format_clean

--- transforms.conf ---
[cisco_wsa_pacfile_drop]
REGEX = ^(.*)pacfile-splunk: Info: (.*)
FORMAT = nullQueue
DEST_KEY = queue

[cisco_wsa_format_clean]
SOURCE_KEY = _raw
REGEX = ^(.*)accesslogs-splunk: Info: (.*)$
FORMAT = $2
DEST_KEY = _raw

Regards, Adriano

[jayannah]

Just change the regex and try

                       [cisco_wsa_pacfile_drop]
                       REGEX = pacfile-splunk 
                       DEST_KEY = queue
                       FORMAT = nullQueue

[azanoli]

It doesn't work also

[azanoli]

There are some errors in the config above, here is the corrected version:

--- props.conf ---
    [cisco:wsa:squid]
    TRANSFORMS-wsa_format = cisco_wsa_pacfile_drop, cisco_wsa_format_clean

--- transforms.conf ---
    [cisco_wsa_pacfile_drop]
    REGEX = ^(.*)pacfile-splunk(.*)$
    FORMAT = nullQueue
    DEST_KEY = queue

    [cisco_wsa_format_clean]
    SOURCE_KEY = _raw
    REGEX = ^(.*)accesslogs-splunk: Info: (.*)$
    FORMAT = $2
    DEST_KEY = _raw