Splunk Enterprise

How to remove everything BEFORE the second comma AND before indexing?

newsplunker1
Path Finder

I have the following events 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958,test,end,2305,2023/05/04

I want to remove everything before the second comma (including the  comma)

Since i dont want it to be indexed , im using the props and transforms on my HF to do that . My regex seems to work but when i try to implement it ,it does not filter anything 

props.conf

[source::/var/log/splunk/IP/syslog.log]
TRANSFORMS-null = remove_before_comma

transforms.conf

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),
DEST_KEY = queue
FORMAT = nullQueue

Here is the regex 

https://regex101.com/r/Lxqgue/1

Any idea why this is not working properly 

Thanks 

Labels (3)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.
0 Karma

newsplunker1
Path Finder

@richgalloway That seems to do the trick ( I ll mark yours as the asnwer ) - Do you have any idea on how to use SED to replace the strings before the comma with a number lets say 0 for example 

this is the original event 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958

 

becomes like

0,0,013001101002958

0 Karma

richgalloway
SplunkTrust
SplunkTrust

SED is search-time rather than index-time, but you could do it with this props:

SEDCMD-replaceUpToComma = s/^([^,]*,[^,]*),/0/
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Fueling your curiosity with new Splunk ILT and eLearning courses

At Splunk Education, we’re driven by curiosity—both ours and yours! That’s why we’re committed to delivering ...

Splunk AI Assistant for SPL 1.1.0 | Now Personalized to Your Environment for Greater ...

Splunk AI Assistant for SPL has transformed how users interact with Splunk, making it easier than ever to ...

Unleash Unified Security and Observability with Splunk Cloud Platform

     Now Available on Microsoft AzureOn Demand Now Step boldly into the AI revolution with enhanced security ...