DQCatalog

Data Catalog

5 datasets · 14 tables · 146 columns · 8,270,952 total rows

Connections

Dataset	Type	Score	7d	Tables	Rows	Tags	Scanned
payments_pg pg-prod-01.us-east-1.k20x.internal	postgresql	91.4%	+0.3%	3	1,832,200	pci_dsshabeas_data_co	14h ago
crm_postgres pg-crm-01.us-east-1.k20x.internal	postgresql	76.8%	-2.1%	3	582,000	gdprhabeas_data_co	14h ago
analytics_orders xy12345.us-east-1.snowflakecomputing.com	snowflake	88.2%	-0.4%	3	5,652,000	soc2	15h ago
compliance_consent sheets.googleapis.com	google_sheets	82.1%	+1.2%	2	8,412	habeas_data_co	17h ago
legacy_mysql mysql-legacy-01.k20x.internal	mysql	64.3%	-5.2%	3	196,340	—	1d ago

Column similarity — MinHash + Jaccard

Pairs with Jaccard ≥ 0.60 flagged as potential duplicates, renames, or shared master data.

Column A	Column B	Jaccard	Method	Notes
crm_postgres.customers.email	payments_pg.merchants.contact_email	0.89	minhash_jaccard	High overlap — likely same email universe. Normalization must be consistent across both.
crm_postgres.customers.document_id	compliance_consent.consent_log.subject_id	0.97	minhash_jaccard	Near-certain match — same national ID number under two column names. Should be modeled as single canonical column.
legacy_mysql.products.sku	analytics_orders.dim_products.product_code	0.94	minhash_jaccard	sku was renamed to product_code in the dbt dim_products model. Confirmed via lineage edge_010.
crm_postgres.customers.customer_id	analytics_orders.dim_customers.customer_id	0.98	minhash_jaccard	Same PK propagated through dbt dim_customers model. Exact match expected.
payments_pg.transactions.customer_id	crm_postgres.customers.customer_id	0.83	minhash_jaccard	83% overlap — transactions table contains some customers not yet in CRM (guest checkouts).
legacy_mysql.products.price_cop	analytics_orders.fct_orders.revenue_local	0.71	embedding_cosine	Semantic similarity: both represent Colombian Peso monetary amounts. Not a direct FK but a structural match.
crm_postgres.customers.city	crm_postgres.addresses.city	0.86	minhash_jaccard	Same city normalization bug in both tables — cleaning one without the other will create inconsistency.
payments_pg.merchants.nit	legacy_mysql.suppliers.supplier_name	0.61	embedding_cosine	Partial match — some suppliers appear in both datasets under different identifiers. Candidate for master data management.