Hi,
We have a large amount of data in /opt/app/axtract_fe1/var/log/apache2/main_collector_access-*.log file, and we do not want HTTP 200, 204 or 401 logs.
How do I filter this out from being indexed?
//SAMPLE LOG
70.166.76.65 - - [27/Oct/2021:12:42:56 -0400] "POST / HTTP/1.1" 200 2949 "-" "-" R:1 Conn:- PID:12954 RD:45125 CSt:+ FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/45125
70.166.76.65 - - [27/Oct/2021:12:42:56 -0400] "POST / HTTP/1.1" 204 248 "-" "-" R:1 Conn:close PID:12954 RD:40522 CSt:- FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/40522
70.166.76.65 - - [27/Oct/2021:12:43:03 -0400] "POST / HTTP/1.1" 200 800 "-" "-" R:0 Conn:- PID:12945 RD:34579 CSt:+ FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/34579
70.166.76.65 - - [27/Oct/2021:12:43:03 -0400] "POST / HTTP/1.1" 200 2949 "-" "-" R:1 Conn:- PID:12945 RD:43790 CSt:+ FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/43790
70.166.76.65 - - [27/Oct/2021:12:43:03 -0400] "POST / HTTP/1.1" 204 248 "-" "-" R:1 Conn:close PID:12945 RD:40819 CSt:- FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/40819
//Props.conf file
[source::/path/to/your/access.log*] TRANSFORMS-null= setnull
Hi
you need to add transforms.conf file where you are defining REGEX which match those lines which you want discard. Here is some examples
You could test your REGEX with regex101.com to work or use Splunk’s rex command in gui.
R. Ismo