Matching the fields on 2 hosts

[Hawkins]

Being fairly new to many features in Splunk, I wish to verify that the fields on 2 different hosts match for consistency. Here's a simple search to show the fields I'd like to verify.  What's the best way to go about this?

index="postgresql" sourcetype="postgres" host=FLSM-ZEUS-PSQL-*
| table host, node_name, node_id, active, type
| where NOT isnull(node_name)

host	node_name	node_id	active	type
FLSM-ZEUS-PSQL-02	flsm-zeus-psql-02	2	t	standby
FLSM-ZEUS-PSQL-02	flsm-zeus-psql-01	1	t	primary
FLSM-ZEUS-PSQL-01	flsm-zeus-psql-02	2	t	standby
FLSM-ZEUS-PSQL-01	flsm-zeus-psql-01	1	t	primary

[richgalloway]

Which fields are you trying to match?  What exactly does "verify" mean in this context?

[hawkik1]

My goal is to make sure that databases on 2 servers have the same data. I'll be using this search in an alert to monitor the health of a sql cluster. My goal it to create an alert that triggers when  the fields:  node_name, node_id, active or type  on both servers don't match each other.

[richgalloway]

Please explain this use case more.  You say you're looking for matches, but the example output contains 4 unique results.  What is expected to match in that?  Please provide a sample match.

[hawkik1]

I have 2 servers FLSM-ZEUS=-PSQL-01, FLSM-ZEUS-PSQL-02. Both servers are part of a SQL cluster. They both have identical records on them. The fields on both servers are node_name, node_id, active, and type. What I wish to do is come up with a search that makes sure the fields on both servers match. Some of them are multivalue fields. The reason for this is, if the cluster isn't communicating correctly, the records may become out of sync. If this happens, I'll create an alert letting me know.

[richgalloway]

There you go again using the word "match" without explaining what constitutes a match in this use case.   How would a human know if there is a match or not?  Once we know that then we can try to figure out how to get Splunk to make the same determination.

[hawkik1]

Match = When the same field on both hosts has the same value. In the example below, both server1 and server2 have a value of "1" in Field_a. That constitutes a match. If Field_a on both hosts has a value of "1" then we have a match.

Server1 - Field_a=1

Server2 - Field_a=1

I wish to verify that the values in each of the four fields on server1 match the values in each of the 4 fields on server2.

Server1                Server2

node_name  =  node_name

node_id      =     node_id

active        =        active

type         =          type

[richgalloway]

See if this helps.  It groups results by host, node_name, node_id, active, and type.  If there are 2 in a group then it's a match; otherwise, it isn't.

index="postgresql" sourcetype="postgres" host=FLSM-ZEUS-PSQL-*
| fields host, node_name, node_id, active, type
| where NOT isnull(node_name)
| stats count by host, node_name, node_id, active, type
| eval match = if(count=2, "Yes", "No")
| fields - count