Splunk Enterprise

How to remove everything BEFORE the second comma AND before indexing?

newsplunker1
Path Finder

I have the following events 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958,test,end,2305,2023/05/04

I want to remove everything before the second comma (including the  comma)

Since i dont want it to be indexed , im using the props and transforms on my HF to do that . My regex seems to work but when i try to implement it ,it does not filter anything 

props.conf

[source::/var/log/splunk/IP/syslog.log]
TRANSFORMS-null = remove_before_comma

transforms.conf

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),
DEST_KEY = queue
FORMAT = nullQueue

Here is the regex 

https://regex101.com/r/Lxqgue/1

Any idea why this is not working properly 

Thanks 

Labels (3)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The existing transform moves any event that matches the regex to nullQueue.  Try these settings to re-write the event.

[remove_before_comma]
REGEX = ^([^,]*,[^,]*),(.*)
DEST_KEY = _raw
FORMAT = $2
---
If this reply helps you, Karma would be appreciated.
0 Karma

newsplunker1
Path Finder

@richgalloway That seems to do the trick ( I ll mark yours as the asnwer ) - Do you have any idea on how to use SED to replace the strings before the comma with a number lets say 0 for example 

this is the original event 

<190>May 4 20:20:36 data.test.com 1,2023/05/04 20:20:35,013001101002958

 

becomes like

0,0,013001101002958

0 Karma

richgalloway
SplunkTrust
SplunkTrust

SED is search-time rather than index-time, but you could do it with this props:

SEDCMD-replaceUpToComma = s/^([^,]*,[^,]*),/0/
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...