Splunk Enterprise

How to remove everything BEFORE the second comma AND before indexing?

newsplunker1
Path Finder

I have the following events 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958,test,end,2305,2023/05/04

I want to remove everything before the second comma (including the  comma)

Since i dont want it to be indexed , im using the props and transforms on my HF to do that . My regex seems to work but when i try to implement it ,it does not filter anything 

props.conf

[source::/var/log/splunk/IP/syslog.log]
TRANSFORMS-null = remove_before_comma

transforms.conf

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),
DEST_KEY = queue
FORMAT = nullQueue

Here is the regex 

https://regex101.com/r/Lxqgue/1

Any idea why this is not working properly 

Thanks 

Labels (3)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.
0 Karma

newsplunker1
Path Finder

@richgalloway That seems to do the trick ( I ll mark yours as the asnwer ) - Do you have any idea on how to use SED to replace the strings before the comma with a number lets say 0 for example 

this is the original event 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958

 

becomes like

0,0,013001101002958

0 Karma

richgalloway
SplunkTrust
SplunkTrust

SED is search-time rather than index-time, but you could do it with this props:

SEDCMD-replaceUpToComma = s/^([^,]*,[^,]*),/0/
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...