Splunk Enterprise

How to remove everything BEFORE the second comma AND before indexing?

newsplunker1
Path Finder

I have the following events 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958,test,end,2305,2023/05/04

I want to remove everything before the second comma (including the  comma)

Since i dont want it to be indexed , im using the props and transforms on my HF to do that . My regex seems to work but when i try to implement it ,it does not filter anything 

props.conf

[source::/var/log/splunk/IP/syslog.log]
TRANSFORMS-null = remove_before_comma

transforms.conf

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),
DEST_KEY = queue
FORMAT = nullQueue

Here is the regex 

https://regex101.com/r/Lxqgue/1

Any idea why this is not working properly 

Thanks 

Labels (3)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.
0 Karma

newsplunker1
Path Finder

@richgalloway That seems to do the trick ( I ll mark yours as the asnwer ) - Do you have any idea on how to use SED to replace the strings before the comma with a number lets say 0 for example 

this is the original event 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958

 

becomes like

0,0,013001101002958

0 Karma

richgalloway
SplunkTrust
SplunkTrust

SED is search-time rather than index-time, but you could do it with this props:

SEDCMD-replaceUpToComma = s/^([^,]*,[^,]*),/0/
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...