Splunk Search

How to join two with datasets with a common field using appendcol ?

zacksoft_wf
Contributor

I have a lookup | inputlookup citizen_data , it has fields ID, Name, State.
I have another sourcetype | index=bayseian souretype=herc , that has fields citizen_ID, mobile, email.

My target is to enrich the "citizen_data" lookup with additional columns so that, while doing  |inputlookup citizen_data, I should see ID, Name, State, Mobile, Email.

NOTE :  ID field in the lookup is same as citizen_ID field in the sourcetype and I wanted the appendcol to link properly as per the matching ID data.

But when I executed my query with appendcol, it does append new columns/fields in the lookup, but it doesn't link or match them comparing  the common field i.e.  the ID. It just randomly appends the new coloumns. The rows contain incorrect data.

Any suggestion  how to append properly by  the common field (ID/citizen_ID)?

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

I said append not appendcols!

| inputlookup citizen_data 
| eval ID1 = ID 
| rename ID1 as citizen_ID 
| stats values(name) as name, values(state) as state  by citizen_ID 
| append 
  [ search index=bayseian souretype=herc 
  | stats values(mobile) as mobile, values(email) as email by citizen_ID ]
| stats values(name) as name, values(state) as state values(mobile) as mobile, values(email) as email  by citizen_ID 

View solution in original post

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Either use join or append (rows of events) and then gather using stats by ID. In both case you will need to rename the common field to be the same name

zacksoft_wf
Contributor

I did rename, but it didn't append based on the common field it just appended randomly Here is the query ================================ | inputlookup citizen_data | eval ID1 = ID | rename ID1 as citizen_ID | stats values(name) as name, values(state) as state  by citizen_ID | appendcols [ search index=bayseian souretype=herc | stats values(mobile) as mobile, values(email) as email by citizen_ID ] | table ID, name, state, mobile, email ============================================

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

I said append not appendcols!

| inputlookup citizen_data 
| eval ID1 = ID 
| rename ID1 as citizen_ID 
| stats values(name) as name, values(state) as state  by citizen_ID 
| append 
  [ search index=bayseian souretype=herc 
  | stats values(mobile) as mobile, values(email) as email by citizen_ID ]
| stats values(name) as name, values(state) as state values(mobile) as mobile, values(email) as email  by citizen_ID 
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...