How to Parse and Tag custom application logs before forwarding to Splunk server?

[filosv]

Dear Splunkers,

really sorry for my question , I do feel that reply would be on another thread(couldn't find it), but i try to forward custom application access logs to  Splunk, giving specific tag-name to each column let's say(i would define it by regular expression), sending only "matching" data. I 've already set inputs.conf with the file path, index and sourcetype and successfully see full logs on Splunk search but whole info on event data . Still not sure where to set appropriate configuration(props.conf, tranform.conf, ?) for getting only Invoked Service, Caller IP and Response Code let's say since we are referring to Access Logs. 

[filosv]

First thanks for your contribution and prompt response. As you might see on the image below there are common access log info like date, time, source IP, duration, response code, which i would like to tag on forwarder side(have no access on Splunk Server), before reaching Splunk server and get rid of unwanted info. What i have only done is adding below lines on my local inputs.conf

index = ....
[monitor://...]
sourcetype = ...
queueSize = 50MB
crcSalt = <SOURCE>
disabled = false

[richgalloway]

I read the OP as wanting to change the data before it is indexed.  I understand now it must be done before sending to the indexers.

If you are using a heavy forwarder then SEDCMD props.conf still is an option.

If you are using Universal Forwarders then there is little the UF can do to modify the data.  Try these untested settings on the UF:

[gr1347yr_access_logs]
force_local_processing = true
SEDCMD-no_foo = s/foo=bar//g

[filosv]

Thanks a lot for your help. Really appreciate it. Got to know how to get rid of unwanted data 😉

[richgalloway]

It would help to see some sample sanitized events, but you may be able to use SEDCMD to eliminate unwanted fields.

props.conf:

[mysourcetype]
SEDCMD-erase_foo = s/foo=bar//

Also, consider using Cribl (cribl.io).