Screening engines and match resolution
Build a reliable list pipeline and an explainable review process.
The name is spelled three ways in three systems. One version loses an accent; another reverses the family name. Screening is a data-engineering problem before it is a matching problem.
Version the list pipeline
A screening decision depends on the list data used at that time. Track source, retrieval time, publication version where available, integrity checks, and activation time. A download that succeeds does not prove the file was parsed completely or deployed to every worker.
Use a staged activation: fetch, validate, compare, approve according to policy, activate, and confirm coverage. Preserve prior versions for replay and investigation. Alert on stale data, malformed records, unexpected count changes, and inconsistent worker versions. Do not silently continue forever with an old list after an update failure.
List ingestion is a production dependency. The system needs evidence that it obtained the expected release, parsed it correctly, loaded the intended records, and activated the right version. A successful download can still contain an unexpected format or an incomplete file. Compare counts and structural properties, preserve the source version, and define a controlled response when validation fails.
Screening results should identify the list and matching logic used at the decision time. Without that information, a later reviewer may replay an old payment against today’s data and mistake a changed result for an earlier system error. A current rescreen and a historical reconstruction are both useful, but they answer different questions and need separate labels.
- FetchRetrieve the authoritative data
- ValidateCheck integrity and record changes
- ActivateConfirm the version used by workers
- Downloaded version
- File reached the pipeline
- Active version
- Matching workers actually use it
List deployment
Illustrative data; not a real customer record or a prescribed policy.
- Fetchedv102
New source data
- Validatedv102
Parsing passed
- Worker group Bv101
Partial activation remains
A successful download is not deployment completion
Verify active versions across workers. A successful download is not deployment completion.
- Failure mode 1avoid
- Overwrite the prior list without history. Past decisions become hard to reproduce.
- Failure mode 2avoid
- Ignore stale-list alerts. The control may miss new restrictions.
- Failure mode 3avoid
- Accept an empty parse as a clean list. It may be a pipeline failure.
Normalize without destroying identity
Normalization can standardize spacing, case, punctuation, and script handling. Preserve original names and identifiers. Some transformations improve matching; others collapse distinct names or lose useful information.
Test transliteration, name order, aliases, organization suffixes, and non-Latin scripts using representative fixtures. Keep the normalization version with the result. A matching change can alter both missed matches and false positives. Do not assume English name structure applies everywhere. The review screen should show the original source strings alongside normalized forms so the analyst can inspect the evidence.
- PreserveKeep original identity strings
- NormalizeApply documented transformations
- CompareUse aliases and relevant identifiers
- Normalized form
- Representation for comparison
- Original form
- Source evidence with its original detail
Name processing
Illustrative data; not a real customer record or a prescribed policy.
- Originalaccented source name
Retained evidence
- Normalizedstandard comparison form
Search input
- Versionnormalizer-v4
Reproducible transformation
Normalization can change identity meaning
Preserve originals and version transformations. Normalization can change identity meaning.
- Failure mode 1avoid
- Discard original script. Review loses useful evidence.
- Failure mode 2avoid
- Assume all names have one first and last name. Name structures vary.
- Failure mode 3avoid
- Tune only on English examples. Coverage may fail for other scripts.
Use more than name similarity
A similarity score ranks candidate strings. It is not the probability that a person is sanctioned unless a separate validated model establishes that meaning. Dates, addresses, identifiers, aliases, and other relevant attributes can help resolve identity.
Define which mismatches are reliable and which fields can be incomplete. A missing birth date is not the same as a conflicting verified birth date. Record the evidence used for resolution and avoid a single unexplained false-positive button. Review decisions should be reproducible from the list record and customer evidence available then.
- CandidateGenerate plausible matches
- IdentifiersCompare supporting attributes
- ResolutionRecord the evidence and reason
- Missing identifier
- No value to compare
- Conflicting identifier
- Evidence supports a different value
Match resolution
Illustrative data; not a real customer record or a prescribed policy.
- Namesimilar
Candidate generated
- Identifierverified different number
Distinguishing evidence
- Reasonseparate entity
Documented resolution
Name similarity alone is not a legal identity finding
Resolve with identifying evidence. Name similarity alone is not a legal identity finding.
- Failure mode 1avoid
- Call a score a probability without validation. The scale may only rank strings.
- Failure mode 2avoid
- Treat missing as conflicting. Absence is not disagreement.
- Failure mode 3avoid
- Allow unexplained dismissals. The result cannot be reviewed.
Control suppression and re-screening
A resolved false match can reduce repeated work if the evidence and list context remain applicable. Suppression should be specific, documented, and reviewable. A blanket allowlist on a common name can hide a future relevant match.
Define when a prior resolution expires or must be reconsidered: changed customer data, changed list record, new ownership evidence, or another relevant trigger. Re-screen the required populations after material updates. Preserve why an alert was suppressed and what evidence supported the reuse. The system must be able to distinguish a reused valid resolution from an alert that was never evaluated.
Suppression reduces repeated work when a prior review has established a valid reason to clear a recurring match. Its scope matters. A clearance for one customer and one list entry should not silently clear every customer with a similar name. Bind the suppression to the evidence, relevant identifiers, reviewer, date, and conditions that invalidate it. Changes to the list, customer identity, or ownership can require a new assessment. The queue becomes quieter for a defensible reason rather than because yesterday’s conclusion became an unlimited exemption.
- ResolveStore the specific false-match basis
- ReuseApply only within the valid scope
- Re-screenTrigger on relevant changes
- Specific suppression
- Customer-list pair with supporting evidence
- Blanket allowlist
- Broad bypass without sufficient scope
Suppression record
Illustrative data; not a real customer record or a prescribed policy.
- Customerentity-72
Specific subject
- List recordtarget-18
Specific candidate
- Triggerlist identity changed
Requires reconsideration
New evidence can change the match
Scope and expire reused resolutions. New evidence can change the match.
- Failure mode 1avoid
- Allowlist every similar name. That can hide another person.
- Failure mode 2avoid
- Suppress without an audit record. The decision is not explainable.
- Failure mode 3avoid
- Skip re-screening after relevant changes. The old conclusion may no longer apply.
Test failures and release controls
Screening outages need a defined product response. Mandatory controls cannot be bypassed merely to preserve conversion. A preapproved contingency might stop new releases, queue eligible work, or route to an alternate controlled process, depending on the legal and operational context.
Test timeouts, stale lists, partial updates, queue failures, and unauthorized release attempts. Keep manual release authority limited and logged. Reconcile all transactions held during an outage with their eventual disposition. A system that recovers technically but loses the held population has not recovered operationally.
- FailDetect the control outage
- ContainApply the approved contingency
- RecoverReconcile held work and dispositions
- Service restored
- Screening responds again
- Operational recovery
- Every held item has a supported outcome
Outage recovery
Illustrative data; not a real customer record or a prescribed policy.
- Held payments240
Preserved population
- Resolved238
Completed dispositions
- Remaining2
Recovery not yet complete
Restored uptime does not resolve missing work
Reconcile all held transactions after recovery. Restored uptime does not resolve missing work.
- Failure mode 1avoid
- Approve all during timeout. That can bypass required controls.
- Failure mode 2avoid
- Give every agent release authority. Sensitive overrides need restricted ownership.
- Failure mode 3avoid
- Drop the queue on restart. The obligations and decisions remain.
Chapter connections
This chapter builds on Sanctions scope, prohibitions, and licenses. Continue with Ownership graphs and the 50 Percent Rule to follow the next part of the system. Use the glossary for terminology and risk mathematics for formulas and worked calculations.