Skip to content
Resistance Kitty home

Resistance Kitty

The sassiest cat fighting fascism

  • Home
  • About Kitty
  • News & Assignments
  • Comics
  • Resistance Survival Guides
  • Sleuth Survival Guides
  • Resistance Directory
  • Merch & Mayhem
  • Toggle search form
RSG 333: Audit secret government watchlists and risk scores. Examine systems, data, authority, errors, bias and challenges.

RSG#333: How To Audit Secret Government Watchlists and Risk Scores

Posted on August 18, 2026August 18, 2026 Resistance Kitty By Resistance Kitty No Comments on RSG#333: How To Audit Secret Government Watchlists and Risk Scores

Secret government watchlists and risk scores can influence who gets stopped, searched, questioned, investigated, denied a benefit, or quietly assigned extra scrutiny. Yet the affected person may never see the record, learn the rule, or receive a useful explanation. Nothing says procedural fairness quite like being judged by a spreadsheet nobody admits exists.

Auditing these systems requires reconstructing the machinery around them. This guide explains how to audit government watchlists and risk scores by identifying their data sources, rules, users, vendors, consequences, error controls, and correction processes. The goal is accountability without exposing private information or helping anyone evade lawful screening. Secrecy may protect particular intelligence. It should not become a scented candle placed over every bad database decision.

Understand What You Are Auditing

A watchlist may be an identity database feeding several screening programs, a local gang database, a threat file, or flags inside another system. A risk score is not necessarily artificial intelligence. It may come from a statistical model, vendor formula, point system, or ordinary rules wearing an expensive technology badge.

Keep the layers separate: source information, inclusion criteria, matching or scoring, and the final government action. An error at any layer can travel downstream and acquire official confidence. The FBI says its renamed Threat Screening Center maintains the consolidated federal terrorism watchlist and shares information with screening partners. Still, an airport delay does not prove someone is listed, and unrelated government scores do not automatically belong to that system.

Step by Step Guide

Step One: Define the Government Action

Begin with the consequence you can observe. Record the date, agency, location, notice, questions, delay, denial, referral, or added condition. Save letters, portal messages, screenshots, receipts, and appeal instructions. Begin with the decision. Evidence enjoys arriving before adjectives.

Step Two: Identify the System and Its Owner

Search budgets, meeting records, manuals, audits, procurement files, and job postings for the program name. Record the data owner, users, vendors, former names, and abbreviations. The office that delivered the bad news may not control the underlying data.

Step Three: Find the Legal Authority

Locate the law or written policy authorizing the system and its consequences. Identify the permitted purpose, covered population, decision maker, review schedule, and sharing limits. If the agency invokes public safety but cannot identify authority for the actual use, flag the gap.

Step Four: Read the Privacy Notices

For federal systems, search privacy pages and the Federal Register for a System of Records Notice. The Justice Department Privacy Act overview explains that these notices describe systems retrieving records by a personal identifier. Read the populations, sources, routine uses, retention rules, safeguards, and access procedures. Find Privacy Impact Assessments and later amendments, then compare the disclosed purpose with actual use.

Step Five: Map Every Data Source

Map where each field originates, including government records, tips, interviews, social media, commercial databases, sensors, and partner agencies. Record whether each source is verified, updated, and correctable. A model can perform flawless mathematics on rotten inputs and still produce premium grade nonsense.

Step Six: Trace the Vendor and Contract

Search SAM.gov contracting records and USAspending.gov for the product, vendor, parent company, award number, and subcontractors. Locally, search board packets, bid portals, payment registers, and sole source justifications. Request the contract, statement of work, data rights clauses, validation reports, training materials, and amendments. Proprietary software does not turn public authority into private weather.

Step Seven: Reconstruct Inclusion and Scoring Rules

Request policies describing who may nominate a person, what facts qualify, what thresholds mean, and whether a human reviews the result. Seek field definitions, manuals, decision tables, validation plans, version histories, and change logs. If operational details are protected, request non sensitive portions and aggregate descriptions. Your audit needs accountability rules, not a road map for evasion.

Step Eight: Follow the Score Into the Decision

Determine whether the score informs an official or automatically triggers action. Compare policies with case records, hearing transcripts, and notices. A score may be marketed as advisory while functioning as an order because nobody wants to disagree with the computer.

Step Nine: Test Accuracy and Disparate Impact

Request aggregate data on entries, removals, false matches, overrides, complaints, outcomes, and demographics. Seek validation studies separating false positives from false negatives across relevant groups. The NIST AI Risk Management Framework treats validity, transparency, accountability, privacy, and harmful bias as distinct concerns. Overall accuracy can conceal unequal mistakes.

Step Ten: Audit Retention and Sharing

Identify how long records remain active, what triggers review, and whether corrections reach every recipient. Request retention schedules, purge logs, access logs, sharing agreements, and audits. The California State Auditor found unsupported entries and defective purge dates in its CalGang audit, reminding us that deletion rules matter only when databases obey them.

Step Eleven: Test Notice and Redress

Document the notice, reviewable evidence, appeal official, deadlines, and correction process. For repeated travel screening problems, the DHS Traveler Redress Inquiry Program is the relevant federal route, not a universal watchlist appeal. The Privacy Act may permit some United States citizens and lawful permanent residents to seek access or amendment, but exemptions can restrict those rights.

Step Twelve: File Focused Records Requests

Ask for existing records, not explanations the agency would have to create. Identify the system, office, dates, document types, contract numbers, and policy versions. Seek aggregate statistics, native electronic files, segregable portions, and the exemption applied to each withholding. The official FOIA guide says a request must reasonably describe agency records. Appeal inadequate searches and unsupported secrecy on time.

Step Thirteen: Reproduce and Publish the Audit

Maintain a source log for every claim. Label confirmed facts, comparisons, expert interpretations, and unresolved inferences. Publish your data dictionary, calculations, exclusions, and limitations without exposing private people. The Electronic Frontier Foundation and American Civil Liberties Union offer independent research on transparency and due process. Use it to supplement primary records.

Red Flags That Deserve Closer Attention

Watch for undocumented criteria, missing privacy notices, unexplained sources, unrecorded model changes, permanent consequences from temporary information, repeated false matches, inaccessible appeals, and corrections that never reach partner agencies. Pay special attention when the agency cannot state how many people are affected or how the tool performs after deployment. A dashboard with three green circles is office decor with electricity.

What Secrecy Does and Does Not Prove

Agencies may lawfully withhold classified information, active investigative material, protected methods, or private data. Withholding does not prove the system is accurate or authorized. Request policies, aggregates, audits, contracts, retention rules, and segregable portions. Never publish names merely because they appear in a leaked or disputed database. Inclusion is not proof of wrongdoing, and accountability reporting should not recreate the harm it investigates.

Turning Findings Into Accountability

Present the audit as a chain from source data to inclusion, score, action, notice, appeal, correction, and deletion. Send findings to the agency privacy office, inspector general, oversight body, public defender, civil liberties group, or independent newsroom. Ask for independent validation, meaningful notice, correction deadlines, review dates, vendor transparency, and public performance reports. A misspelled name needs correction. A structurally biased process may need suspension.

Closing RK Thoughts

Secret government watchlists and risk scores gain power from fragmentation. One office supplies data. Another buys software. A third applies the score. A fourth delivers the consequence. Each points toward the next building until accountability develops shin splints.

A serious audit reconnects those pieces and tests whether mistakes can be fixed before they become permanent government folklore. Resistance Kitty does not expect every sensitive record to be public. She does expect the government to prove that secret systems are lawful, tested, reviewable, and capable of recognizing when they are wrong.

Sources

  1. FBI Threat Screening Center
  2. FBI Announcement of the Threat Screening Center Name Change
  3. Government Accountability Office Review of Terrorist Watchlist Nomination and Redress
  4. Department of Homeland Security Traveler Redress Inquiry Program
  5. Justice Department Privacy Act Overview
  6. Justice Department Privacy Act Agency Requirements
  7. FOIA.gov Request Guide
  8. NIST Artificial Intelligence Risk Management Framework
  9. California State Auditor Review of CalGang
  10. Electronic Frontier Foundation Transparency Project
  11. American Civil Liberties Union Watchlists Project
  12. ProPublica Methodology for Auditing COMPAS

Support Resistance Kitty’s Work

  • Merch & Mayhem
  • Buy Resistance Kitty a Treat
Resistance Survival Guide Tags:algorithmic accountability, DHS TRIP, gang databases, government risk scores, government watchlists, predictive policing, Privacy Impact Assessment, public records investigation, Resistance survival guide, surveillance oversight, System of Records Notice, Threat Screening Center

Post navigation

Previous Post: RSG #332: How To Trace Public Money Through Pass Through Nonprofits and Fiscal Sponsors
Next Post: Day 574 DHS FINANCIAL COLONOSCOPY

Related Posts

  • Resistance Survival Guide banner with Resistance Kitty.
    #147 Tariffs, Torture, and the Supreme Court Circus — Time to Scratch Back Resistance Survival Guide
  • Resistance Survival Guide banner with Resistance Kitty.
    RSG #237: War Prep — Building a Personal Supply Cache That Actually Keeps You Alive Resistance Survival Guide
  • Resistance Survival Guide banner with Resistance Kitty.
    #174 Document Everything Like a Journalist (Because Receipts Beat Rage) Resistance Survival Guide
  • RSG 316: Audit the algorithm. Trace contracts, data, false flags, appeals and who is harmed.
    RSG #316: How To Audit Government Algorithms for Discriminatory Errors Resistance Survival Guide
  • RSG 311: Detect a coordinated government records destruction pattern by comparing retention, deletion, backups and audit records.
    RSG #311: How To Detect A Coordinated Government Records Destruction Pattern Resistance Survival Guide
  • RSG 305: Track real estate power around public projects. Follow plans, deeds, zoning, taxes and beneficiaries.
    RSG #305: Tracking Real Estate Power Around Public Projects Resistance Survival Guide

More Related Articles

RSG 260: Protect mental health in an overwhelming environment by setting boundaries and taking small actions. RSG #260: How to Protect Your Mental Health in an Environment Designed to Overwhelm You Resistance Survival Guide
Resistance Survival Guide banner with Resistance Kitty. #163 Digital & Community Organizing Under Pressure Resistance Survival Guide
Resistance Survival Guide banner with Resistance Kitty. RSG #215 How to Find Who Funds a Political Candidate Resistance Survival Guide
RSG #354: How To Reconstruct Deleted or Missing Government Communications From Secondary Records RSG #354: How To Reconstruct Deleted or Missing Government Communications From Secondary Records Resistance Survival Guide
Resistance Survival Guide banner with Resistance Kitty. #174 Document Everything Like a Journalist (Because Receipts Beat Rage) Resistance Survival Guide
RSG 308: Reconstruct a government algorithmic decision system. Trace contracts, data, rules, human overrides and appeals. RSG #308: How To Reconstruct A Government Algorithmic Decision System Resistance Survival Guide

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Resistance Kitty in a dark suit

Resistance Kitty

Satire. Survival. Solidarity.

News with receipts. Tools for fighting back. A cat with absolutely no patience for fascism.

Meet Resistance Kitty

Find the Receipts

Keep Your Claws Out

  • News and Assignments What happened and what you can do.
  • Resistance Survival Guides Practical steps to investigate, organize and protect yourself.
  • Kitty’s Comics Because sometimes the punchline has claws.

Kitty in Your Inbox

Get the next briefing, survival guide and dose of righteous cattitude.

Subscribe on Substack

Follow the Work

  • Resistance Directory Find resources, tools and allies.
  • EpsteinWiki Follow the evidence and investigate the records.
  • The Butterfly Bureau Dig deeper into investigations.

Fuel the Cattitude

Help keep Kitty reporting, researching and making useful noise.

Support Resistance Kitty Shop Merch & Mayhem
Resistance Kitty in a dark suit

Resistance Kitty

Satire. Survival. Solidarity.

News with receipts. Practical ways to fight back. Comics for when democracy needs claws.

Explore

  • Home
  • About Resistance Kitty
  • News
  • Calls to Action
  • Survival Guides
  • Comics

Follow the Work

  • EpsteinWiki
  • Resistance Directory
  • The Butterfly Bureau
  • Merch & Mayhem

Keep Your Claws Out

Get the next briefing and help keep Kitty reporting.

Subscribe on Substack Support Resistance Kitty

© 2026 Resistance Kitty. All rights reserved.

Stay informed. Stay loud. Stay connected.

Copyright © 2026 Resistance Kitty.