DQ
DQCatalogCompliance_consent
Catalog/

compliance_consent

Type: google_sheetsHost: sheets.googleapis.comDB: Habeas Data — Consent RegistryLast scan: 17h agoDuration: 14s
habeas_data_co

Overall score

82.1%

+1.2% vs last week

Six-dimension scores

completeness

93.7%

+1.2%

6.3% null rate on consent_timestamp (529 rows). subject_id 100% present.

uniqueness

96.4%

+0.3%

301 apparent duplicate subject+policy combinations — person re-consented under new policy?

validity

89.1%

+0.8%

3 consent_timestamp values are future-dated (data entry error). 14 rows have invalid purpose_code.

consistency

81.2%

+1.4%

Mismatched policy_version_id references: 42 rows reference policy versions not in policy_versions sheet.

integrity

79.5%

+1.8%

529 rows with null consent_timestamp cannot be audited. 42 orphan FK refs to policy_versions.

accuracy

87.8%

+0.7%

Date formats inconsistent (DD/MM/YYYY vs YYYY-MM-DD). Automated parse introduces ±1 day error risk.

compliance_consent.consent_log

8,400 rows · 8 cols · scanned 17h ago
ColumnTypeNull%DistinctMeanp95DimensionsNotes
consent_id
text0%8,400
compl
100.0%
valid
100.0%
uniq
100.0%
subject_idPII

co_cedula

text0%7,728
compl
100.0%
valid
99.6%
uniq
92.0%
Matches crm_postgres.customers.document_id — Jaccard 0.97.
consent_timestamp
text6.3%7,560
compl
93.7%
valid
89.1%
uniq
90.0%
529 null timestamps. CRITICAL for Habeas Data compliance.
purpose_code
text0.3%6
compl
99.7%
valid
99.8%
uniq
0.0%
policy_version_id
text0%14
compl
100.0%
valid
89.0%
uniq
0.0%
42 rows reference policy_version_id not in policy_versions sheet.
channel
text1.0%5
compl
99.0%
valid
100.0%
uniq
0.0%

compliance_consent.policy_versions

12 rows · 5 cols · scanned 17h ago
ColumnTypeNull%DistinctMeanp95DimensionsNotes
policy_version_id
text0%12
compl
100.0%
valid
100.0%
uniq
100.0%
effective_date
text0%12
compl
100.0%
valid
91.7%
uniq
100.0%
1 row has invalid date format.

Auto-prioritized fixes (2)

criticalconsent_log.consent_timestampcompleteness

Backfill 529 null consent_timestamps — Habeas Data critical

529 rows (6.3%) in consent_log have a null consent_timestamp. Colombian Habeas Data law (Ley 1581 de 2012) requires a documented timestamp for every consent record. These rows will fail the next SFC audit.

UPDATE consent_log SET consent_timestamp = created_at WHERE consent_timestamp IS NULL AND created_at IS NOT NULL;
-- Remaining nulls: escalate to operations team for manual backfill from paper forms.

529 rows affected

warningconsent_log.policy_version_idintegrity

Resolve 42 orphan policy_version_id references

42 consent_log rows reference policy_version_id values not present in the policy_versions sheet. This is an orphan FK — either the policy_versions sheet is missing historical entries, or the consent_log contains future-dated or test rows that shouldn't be there.

-- Identify orphans
SELECT cl.consent_id, cl.policy_version_id
FROM consent_log cl
LEFT JOIN policy_versions pv ON cl.policy_version_id = pv.policy_version_id
WHERE pv.policy_version_id IS NULL;
-- Then: backfill missing policy_versions rows or mark orphan consents as 'pending_review'.

42 rows affected

Column similarity matches

Column AColumn BJaccard
crm_postgres.customers.document_idcompliance_consent.consent_log.subject_id0.97