I have the following events
<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958,test,end,2305,2023/05/04
I want to remove everything before the second comma (including the comma)
Since i dont want it to be indexed , im using the props and transforms on my HF to do that . My regex seems to work but when i try to implement it ,it does not filter anything
props.conf
[source::/var/log/splunk/IP/syslog.log]
TRANSFORMS-null = remove_before_comma
transforms.conf
[remove_before_comma]
REGEX = ^([^,]*,[^,]*),
DEST_KEY = queue
FORMAT = nullQueue
Here is the regex
https://regex101.com/r/Lxqgue/1
Any idea why this is not working properly
Thanks
The existing transform moves any event that matches the regex to nullQueue. Try these settings to re-write the event.
[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
The existing transform moves any event that matches the regex to nullQueue. Try these settings to re-write the event.
[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
@richgalloway That seems to do the trick ( I ll mark yours as the asnwer ) - Do you have any idea on how to use SED to replace the strings before the comma with a number lets say 0 for example
this is the original event
<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958
becomes like
0,0,013001101002958
SED is search-time rather than index-time, but you could do it with this props:
SEDCMD-replaceUpToComma = s/^([^,]*,[^,]*),/0/