Splunk Search

Remove a series of numbers in a field

AccentureQBETA
Path Finder

I have a field which is extracted in Splunk with values which look like this:

/aa/Application.do?inFrame=uploadframe&r=99946238&__navigator_index=0
/aa/resources/Ocean/css/trans.css?ver=6.0.4.21
/aa/Application.do?inFrame=scframe&r=99989045&__navigator_index=0
/aa/Application.do?inFrame=blank&r=99985838&__navigator_index=0
/aa/soap/Something
/aa/resources/Ocean/css/print_trans.css?ver=6.0.4.21
/aa/?internal=Y
/aa/Application.do?ts=99998203
/aa/soap/ILoginAPI
/aa/FileContents.do?widgetID=Application:Something_Something:Ext_Something:9:Something_ViewLink

I would like to the remove the series of numbers, where there are more than 3 in a row.

I think it can be done either at index time or search time using rex? I can't figure it out though..

1 Solution

echalex
Builder

I think it could be done using index-time, but it's probably a better idea to do it search-time by using eval and replace.

...|eval your_field=replace(your_field,"\d{3}\d+","")

(Assuming that by "more than 3" you mean "four or more" and not "three or more".)

View solution in original post

echalex
Builder

I think it could be done using index-time, but it's probably a better idea to do it search-time by using eval and replace.

...|eval your_field=replace(your_field,"\d{3}\d+","")

(Assuming that by "more than 3" you mean "four or more" and not "three or more".)

AccentureQBETA
Path Finder

This is perfect, thanks

Jason
Motivator

Doing it at index time or search time depends on whether you want the data to be permanently modified in your index or not.

At index time, you can use the SEDCMD to rewrite the data:
props.conf

[datasource]
SEDCMD-removelongnumbers = s/(.*?)\d\d\d+(.*)/\1\2/

Or, at search time, you could use "rex" and "eval" to remove them, but the other answer is easier:

... | rex field=myurl "(?<myurl1>.*?)\d\d\d+(?<myurl2>.*)" | eval myurl=if(isnull(myurl1), myurl, myurl1+myurl2)

Test these prior to using, I don't have an instance in front of me right now.

0 Karma

Jason
Motivator

Yes, SEDCMD is run on the entire event, since most fields do not exist until search time.

0 Karma

AccentureQBETA
Path Finder

Jason, thank you so much for your answer. I havn't tried them yet, but will today and comment back. Echalex's answer is easier to understand for me, for now and works. Thank you though!

One thing though, because I might use your index-time solution later.. doesn't that do it for the whole event and not just a field? can a n index-time SEDCMD be done on a field? maybe if the stanza comes after a field extract stanza?

0 Karma

AccentureQBETA
Path Finder

Yeah, that's correct 🙂

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

So you want, e.g., the first few to be:

/aa/Application.do?inFrame=uploadframe&r=&__navigator_index=0
/aa/resources/Ocean/css/trans.css?ver=6.0.4.21
/aa/Application.do?inFrame=scframe&r=&__navigator_index=0
/aa/Application.do?inFrame=blank&r=&__navigator_index=0
/aa/soap/Something
/aa/resources/Ocean/css/print_trans.css?ver=6.0.4.21
...

? Is that correct?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...