Splunk Enterprise

How to remove everything BEFORE the second comma AND before indexing?

newsplunker1
Path Finder

I have the following events 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958,test,end,2305,2023/05/04

I want to remove everything before the second comma (including the  comma)

Since i dont want it to be indexed , im using the props and transforms on my HF to do that . My regex seems to work but when i try to implement it ,it does not filter anything 

props.conf

[source::/var/log/splunk/IP/syslog.log]
TRANSFORMS-null = remove_before_comma

transforms.conf

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),
DEST_KEY = queue
FORMAT = nullQueue

Here is the regex 

https://regex101.com/r/Lxqgue/1

Any idea why this is not working properly 

Thanks 

Labels (3)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.
0 Karma

newsplunker1
Path Finder

@richgalloway That seems to do the trick ( I ll mark yours as the asnwer ) - Do you have any idea on how to use SED to replace the strings before the comma with a number lets say 0 for example 

this is the original event 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958

 

becomes like

0,0,013001101002958

0 Karma

richgalloway
SplunkTrust
SplunkTrust

SED is search-time rather than index-time, but you could do it with this props:

SEDCMD-replaceUpToComma = s/^([^,]*,[^,]*),/0/
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...