All Apps and Add-ons

Unable to delete some logs with TRANSFORMS

azanoli
Explorer

Hi all

I have problems with 2 transforms for a Cisco IronPort Proxy. I receive 2 different types of logs:

Pacfile download:

Feb 3 15:08:22 ironport2.bank.com Feb 03 15:08:22 pacfile-splunk: Info: 189.149.11.14 - /proxy-ntlm.pac is downloaded successfully

Proxy-Access:

Jan 8 09:22:19 proxy.bank.com Jan 08 09:22:19 accesslogs-splunk: Info: 1420705339.401 307 189.149.128.70 TCP_CLIENT_REFRESH_MISS/200 6439 CONNECT tunnel://eqi.ibb.ubs.com:443/ "MAIN\bsm@MAIN.NTLM" DIRECT/eqi.ibb.ubs.com - PASSTHRU_CUSTOMCAT_7-DefaultGroup-ID.G.URL.Domains.Whitelist-DefaultGroup-NONE-NONE-DefaultGroup <C_G.N0,-,-,"-",-,-,-,-,"-",-,-,-,"-",-,-,"-","-",-,-,-,-,"-","-","-","-","-","-",167.79,0,-,"-","-",-,"-",-,-,"-","-"> - "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" - 216.219.78.86 - "G.NoDecryption.BypassScanning" 3438

I want to remove the Pacfile entries with the transform cisco_wsa_pacfile_drop. After them I have to reformat the Proxy-Access logs to delete all entries until the timestamp with the transform cisco_wsa_format_clean.
The second part works well. But the deletion of the Pacfile logs doesn't work.

--- props.conf ---
[cisco:wsa:squid]
TRANSFORMS-wsa_format = cisco_wsa_format_clean,cisco_wsa_format_clean

--- transforms.conf ---
[cisco_wsa_pacfile_drop]
REGEX = ^(.*)pacfile-splunk: Info: (.*)
FORMAT = nullQueue
DEST_KEY = queue

[cisco_wsa_format_clean]
SOURCE_KEY = _raw
REGEX = ^(.*)accesslogs-splunk: Info: (.*)$
FORMAT = $2
DEST_KEY = _raw

Regards, Adriano

0 Karma

jayannah
Builder

Just change the regex and try

                       [cisco_wsa_pacfile_drop]
                       REGEX = pacfile-splunk 
                       DEST_KEY = queue
                       FORMAT = nullQueue
0 Karma

azanoli
Explorer

It doesn't work also

0 Karma

azanoli
Explorer

There are some errors in the config above, here is the corrected version:

--- props.conf ---
    [cisco:wsa:squid]
    TRANSFORMS-wsa_format = cisco_wsa_pacfile_drop, cisco_wsa_format_clean

--- transforms.conf ---
    [cisco_wsa_pacfile_drop]
    REGEX = ^(.*)pacfile-splunk(.*)$
    FORMAT = nullQueue
    DEST_KEY = queue

    [cisco_wsa_format_clean]
    SOURCE_KEY = _raw
    REGEX = ^(.*)accesslogs-splunk: Info: (.*)$
    FORMAT = $2
    DEST_KEY = _raw
0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...

Security Highlights | November 2022 Newsletter

 November 2022 2022 Gartner Magic Quadrant for SIEM: Splunk Named a Leader for the 9th Year in a RowSplunk is ...

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...