Splunk Search

Remove a series of numbers in a field

AccentureQBETA
Path Finder

I have a field which is extracted in Splunk with values which look like this:

/aa/Application.do?inFrame=uploadframe&r=99946238&__navigator_index=0
/aa/resources/Ocean/css/trans.css?ver=6.0.4.21
/aa/Application.do?inFrame=scframe&r=99989045&__navigator_index=0
/aa/Application.do?inFrame=blank&r=99985838&__navigator_index=0
/aa/soap/Something
/aa/resources/Ocean/css/print_trans.css?ver=6.0.4.21
/aa/?internal=Y
/aa/Application.do?ts=99998203
/aa/soap/ILoginAPI
/aa/FileContents.do?widgetID=Application:Something_Something:Ext_Something:9:Something_ViewLink

I would like to the remove the series of numbers, where there are more than 3 in a row.

I think it can be done either at index time or search time using rex? I can't figure it out though..

1 Solution

echalex
Builder

I think it could be done using index-time, but it's probably a better idea to do it search-time by using eval and replace.

...|eval your_field=replace(your_field,"\d{3}\d+","")

(Assuming that by "more than 3" you mean "four or more" and not "three or more".)

View solution in original post

echalex
Builder

I think it could be done using index-time, but it's probably a better idea to do it search-time by using eval and replace.

...|eval your_field=replace(your_field,"\d{3}\d+","")

(Assuming that by "more than 3" you mean "four or more" and not "three or more".)

AccentureQBETA
Path Finder

This is perfect, thanks

Jason
Motivator

Doing it at index time or search time depends on whether you want the data to be permanently modified in your index or not.

At index time, you can use the SEDCMD to rewrite the data:
props.conf

[datasource]
SEDCMD-removelongnumbers = s/(.*?)\d\d\d+(.*)/\1\2/

Or, at search time, you could use "rex" and "eval" to remove them, but the other answer is easier:

... | rex field=myurl "(?<myurl1>.*?)\d\d\d+(?<myurl2>.*)" | eval myurl=if(isnull(myurl1), myurl, myurl1+myurl2)

Test these prior to using, I don't have an instance in front of me right now.

0 Karma

Jason
Motivator

Yes, SEDCMD is run on the entire event, since most fields do not exist until search time.

0 Karma

AccentureQBETA
Path Finder

Jason, thank you so much for your answer. I havn't tried them yet, but will today and comment back. Echalex's answer is easier to understand for me, for now and works. Thank you though!

One thing though, because I might use your index-time solution later.. doesn't that do it for the whole event and not just a field? can a n index-time SEDCMD be done on a field? maybe if the stanza comes after a field extract stanza?

0 Karma

AccentureQBETA
Path Finder

Yeah, that's correct 🙂

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

So you want, e.g., the first few to be:

/aa/Application.do?inFrame=uploadframe&r=&__navigator_index=0
/aa/resources/Ocean/css/trans.css?ver=6.0.4.21
/aa/Application.do?inFrame=scframe&r=&__navigator_index=0
/aa/Application.do?inFrame=blank&r=&__navigator_index=0
/aa/soap/Something
/aa/resources/Ocean/css/print_trans.css?ver=6.0.4.21
...

? Is that correct?

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...