Splunk Search

How to join two with datasets with a common field using appendcol ?

zacksoft_wf
Contributor

I have a lookup | inputlookup citizen_data , it has fields ID, Name, State.
I have another sourcetype | index=bayseian souretype=herc , that has fields citizen_ID, mobile, email.

My target is to enrich the "citizen_data" lookup with additional columns so that, while doing  |inputlookup citizen_data, I should see ID, Name, State, Mobile, Email.

NOTE :  ID field in the lookup is same as citizen_ID field in the sourcetype and I wanted the appendcol to link properly as per the matching ID data.

But when I executed my query with appendcol, it does append new columns/fields in the lookup, but it doesn't link or match them comparing  the common field i.e.  the ID. It just randomly appends the new coloumns. The rows contain incorrect data.

Any suggestion  how to append properly by  the common field (ID/citizen_ID)?

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

I said append not appendcols!

| inputlookup citizen_data 
| eval ID1 = ID 
| rename ID1 as citizen_ID 
| stats values(name) as name, values(state) as state  by citizen_ID 
| append 
  [ search index=bayseian souretype=herc 
  | stats values(mobile) as mobile, values(email) as email by citizen_ID ]
| stats values(name) as name, values(state) as state values(mobile) as mobile, values(email) as email  by citizen_ID 

View solution in original post

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Either use join or append (rows of events) and then gather using stats by ID. In both case you will need to rename the common field to be the same name

zacksoft_wf
Contributor

I did rename, but it didn't append based on the common field it just appended randomly Here is the query ================================ | inputlookup citizen_data | eval ID1 = ID | rename ID1 as citizen_ID | stats values(name) as name, values(state) as state  by citizen_ID | appendcols [ search index=bayseian souretype=herc | stats values(mobile) as mobile, values(email) as email by citizen_ID ] | table ID, name, state, mobile, email ============================================

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

I said append not appendcols!

| inputlookup citizen_data 
| eval ID1 = ID 
| rename ID1 as citizen_ID 
| stats values(name) as name, values(state) as state  by citizen_ID 
| append 
  [ search index=bayseian souretype=herc 
  | stats values(mobile) as mobile, values(email) as email by citizen_ID ]
| stats values(name) as name, values(state) as state values(mobile) as mobile, values(email) as email  by citizen_ID 
0 Karma
Get Updates on the Splunk Community!

Update Your SOAR Apps for Python 3.13: What Community Developers Need to Know

To Community SOAR App Developers - we're reaching out with an important update regarding Python 3.9's ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...

Automatic Discovery Part 2: Setup and Best Practices

In Part 1 of this series, we covered what Automatic Discovery is and why it’s critical for observability at ...