topic Splunk Query to extract id's from a event and display the ID's in Splunk Search

Splunk Query to extract id's from a event and display the ID's

Vin — Thu, 06 Mar 2025 20:26:13 GMT

Below is the search and I need to extract the ID's shown in the below event and there are also many other ID's. Please help me in writing a query to extract the ID's which starts with "Duplicate Id's that needs to be displayed ::::::[6523409, 6529865]" in the log file.

index="*" source ="*" "Duplicate Id's that needs to be displayed ::::::[6523409, 6529865]

Re: Splunk Query to extract id's from a event and display the ID's

livehybrid — Thu, 06 Mar 2025 21:21:16 GMT

Hi @Vin

Please could you share some of your raw events so that we can help you further?

In the meantime, you might have some success with something like this?

| rex field=_raw max_match=0 "(?<numbers>\d+)"

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

Re: Splunk Query to extract id's from a event and display the ID's

gcusello — Fri, 07 Mar 2025 07:20:47 GMT

Hi @Vin ,

it's really difficoult to create a regex without a data sample!

anyway, if the ID to extract is the number in the square brackets and you have only one copuple of square brackets you could use this:

| rex "\[(?<your_field>[^\]]+)"

I could be more sure if you can share some data.

Ciao.

Giuseppe

Re: Splunk Query to extract id's from a event and display the ID's

PickleRick — Fri, 07 Mar 2025 12:54:10 GMT

As others already pointed out - what data you're searching?

Also, searching across all events from all indexes will be slooooooow. You should limit your search as early as possible.

But the main issue is - what actually is your problem? You seem to have some search which doesn't actually search for anything, just lists all events. And you want to "display duplicate ids". Do you have statically defined ids? Or do you want to extract the ids from your data and find any that are duplicate? Be more verbose about your problem.

Re: Splunk Query to extract id's from a event and display the ID's

yuanliu — Mon, 10 Mar 2025 04:19:35 GMT

I know it's already a party. But I have to agree with @PickleRick that throwing out a random SPL snippet is not a good way to use volunteers' time. Here are four golden rules of asking an answerable question that I call four commandments:

Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search (SPL that volunteers here do not have to look at).
Illustrate the desired output from illustrated data.
Explain the logic between illustrated data and desired output without SPL.
If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious.

In this spirit, if I have to read your mind, I will start by reverse engineering what your data look like: Some your raw data contains text strings like "Duplicate Id's that needs to be displayed ::::::[6523409, 6529865]". Given such an event, the desired output is a multivalue field containing values 6523409 and6529865. Let us call this field "duplicate_ids". Something to this effect:

_raw	duplicate_ids
blah, blah, blah Duplicate Id's that needs to be displayed ::::::[6523409, 6529865] - and more blahs	6523409 6529865

Is this the use case? If yes, here is what I do

| rex "Duplicate Id's that needs to be displayed :*(?<duplicate_ids>\[[^\]]+\])" | eval duplicate_ids = json_array_to_mv(duplicate_ids)

(The above requires Splunk 8.1 or later. But it is not the only way to do this.)

Here is an emulation for you to play with and compare with real data

| makeresults | fields - _time | eval _raw = "blah, blah, blah Duplicate Id's that needs to be displayed ::::::[6523409, 6529865] - and more blahs" ``` data emulation above ```