Filtering out HTTP/1.1 200 logs from being forwarded to splunk.

[ssoftility]

Hi,

We have a large amount of data in /opt/app/axtract_fe1/var/log/apache2/main_collector_access-*.log file, and we do not want HTTP 200, 204 or 401 logs.

How do I filter this out from being indexed?

//SAMPLE LOG

70.166.76.65 - - [27/Oct/2021:12:42:56 -0400] "POST / HTTP/1.1" 200 2949 "-" "-" R:1 Conn:- PID:12954 RD:45125 CSt:+ FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/45125
70.166.76.65 - - [27/Oct/2021:12:42:56 -0400] "POST / HTTP/1.1" 204 248 "-" "-" R:1 Conn:close PID:12954 RD:40522 CSt:- FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/40522
70.166.76.65 - - [27/Oct/2021:12:43:03 -0400] "POST / HTTP/1.1" 200 800 "-" "-" R:0 Conn:- PID:12945 RD:34579 CSt:+ FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/34579
70.166.76.65 - - [27/Oct/2021:12:43:03 -0400] "POST / HTTP/1.1" 200 2949 "-" "-" R:1 Conn:- PID:12945 RD:43790 CSt:+ FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/43790
70.166.76.65 - - [27/Oct/2021:12:43:03 -0400] "POST / HTTP/1.1" 204 248 "-" "-" R:1 Conn:close PID:12945 RD:40819 CSt:- FT:forwarded CPE_IP:70.166.77.73, 70.166.76.65 RespTime:0/40819

//Props.conf file

[source::/path/to/your/access.log*]
TRANSFORMS-null= setnull

[isoutamo]

Hi

you need to add transforms.conf file where you are defining REGEX which match those lines which you want discard. Here is some examples 

• https://community.splunk.com/t5/Getting-Data-In/Filtering-events-using-NullQueue/m-p/66392
• https://community.splunk.com/t5/Getting-Data-In/How-to-filter-out-debug-logs-except-3-different-log-...
  
  

You could test your REGEX with regex101.com to work or use Splunk’s rex command in gui.

R. Ismo