topic Re: Add field from another index when key value has a different name in Getting Data In

How to add field from another index when key value has a different name?

sergioleone — Wed, 24 May 2023 15:13:46 GMT

I have two indexes and need to pull the idfrom the second into the first. For example I have a log from each index in a table as follows. Index2 has every mail and id for a user meanwhile Index1 has the event data I will be using.

Index: user: mail: id:

index1 name@gmail.com zxshbfbfve093ns
index2 name@gmail.com abc123

I need to pull the id field (abc123) and associate it to the event in index1 where name@gmail.com is matching in index2.

Re: Add field from another index when key value has a different name

gcusello — Wed, 24 May 2023 15:16:34 GMT

Hi @sergioleone ,

you're thinking to Splunk as a DB and to an index as a DB table, but Splunk isn't a DB and an index isn't a table!

In other words you can have etherogenous logs, different in structure and every thing, like the ones you have in the same or different indexes.

Logs are usually stored in an index when they have the same retention period and the same access rules not for their structure or information.

Anyway, you can present as results, without moving a log from an index to another using a search like the following:

index=index1 OR index=index2 | eval user=coalesce(user,mail) | stats values(eval(if(index=index2),id,"") AS id BY user

You could do the thing you would running the above search and saving results in a summary index, it isn't possible to move a part of events from an index to another, but why?

Ciao.

Giuseppe

Re: Add field from another index when key value has a different name

sergioleone — Wed, 24 May 2023 18:17:36 GMT

@gcuselloThank you for your response.

The data shown was when I tabled the index and the other 3 fields to show the fields that I have and what I want to show.

I think there may be a parentheses missing from your stats command as there are 3 on left and only 2 on the right and I get an error message.

Re: Add field from another index when key value has a different name

gcusello — Thu, 25 May 2023 03:59:40 GMT

Hi @sergioleone ,

yes, you're right:

index=index1 OR index=index2 | eval user=coalesce(user,mail) | stats values(eval(if(index=index2),id,"")) AS id BY user

Anyway I hope to have cleared the different Splunk approach: all of us, coming from db development, passed throgh this mind change.

It's the same thing to use the join command: all the people from DB development use join command but join is a very slow command to use only when you haven't any other solution and with very few data, the correct solution is stats command.

Ciao.

Giuseppe