RSG #316: How To Audit Government Algorithms for Discriminatory Errors

Resistance Survival Guide #316

Once upon a time, and not so long ago, a government employee had to look you in the face before making a decision that could ruin your life. But now, the lazy government has decided to let the Cylons take over. Skynet, with little to no human oversight, can flag you as suspicious or dead, remove you from a list, and poof! You are fucked! This can delay your benefits, question your citizenship, or send your name to an investigator before you even know what the actual fuck is happening.

When the system gets it wrong, government officials, if you can find one, will simply blame “the system” and blow off your freak out. Maybe, just maybe, they will fix it, depending on what Skynet tells them, I guess. At this stage, it could take hours, weeks, or months before you get any form of resolution.

The use of AI has already proven that it is a probability machine, not an oracle. We know that generative AI hallucinates and makes shit up. We also know that automated matching systems can confidently flag the wrong person because of outdated, incomplete, or incompatible records. Yet too many government agencies are using these tools as if they are perfect because the government has adopted the capitalist obsession with quantity over quality.

This means we now have to do something most of us have never had to do before. We have to stay on top of our own shit and the government’s shit. We need to know what records exist, what those records say about us, and who is using them. Otherwise, we may not find out there is a problem until some automated system has mistakenly erased us from existence, at least in the eyes of the government. As if we did not have enough shit to do. Eyes rolling.

In the beginning, people chose the information that went into these systems and created the procedures that were supposed to correct the errors. Humans, not Skynet, made the rules, selected the vendors, and established the protocols. An algorithm is not an independent authority. It is a set of human instructions used to process information and produce a result. Anyone who has been on Twitter since Space Karen took over knows that a human ultimately controls the algorithm. Therefore, a human must be responsible for the consequences when it causes harm.

Since the fascist takeover of our government by the broligarchy, federal agencies have been rapidly expanding their use of automated systems and artificial intelligence. These systems can help verify voters, determine eligibility for benefits, identify possible fraud, calculate risk, investigate immigration cases, prioritize inspections, and select people for additional enforcement. Those are only the uses we know about. The Government Accountability Office reported that federal agencies more than doubled their reported use of artificial intelligence from 2023 to 2024.

Not every automated government system uses artificial intelligence. Some rely on formulas, matching rules, or ordinary databases that are far less sophisticated than officials would like us to believe. Artificial intelligence may be one of the worst names ever created for these error driven databases from hell. The name suggests thought, judgment, and understanding. What we often have is a machine identifying patterns without understanding the individual human being who may be destroyed by its conclusion.

The tech bros have forgotten that we are individuals and not machines.

The technology does not need to be advanced to cause advanced levels of damage. We have already watched DOGE driven staffing cuts and technology changes create chaos inside federal agencies. ProPublica reported that experienced Social Security officials watched DOGE prioritize quick wins while ignoring many of the people who understood the agency and its systems. Removing knowledgeable employees while increasing dependence on automated systems is not efficiency. It is an excellent way to create errors while also eliminating the humans who know how to fix them.

In February 2026, ProPublica reported that the expanded federal SAVE citizenship verification system had incorrectly flagged eligible voters. In Boone County, Missouri, officials reviewed 74 people identified as possible noncitizens. More than half were reportedly citizens. One of the flagged voters had been registered during a naturalization ceremony.

Think about that for a moment. The government helped a new citizen register to vote and then another part of the government questioned whether that same person was a citizen. This kitties, is what happens when a database is treated as more credible than the human being standing in front of it, a complete and utter shit show.

The good news is that you do not need a degree in computer science to investigate these systems of chaos and kuntery. You need to understand what decision is being made, what records are being used, how often the system is wrong, and which humans are expected to absorb those errors.

We also need to stop being intimidated by the word “algorithm” and the concept of “AI”. Complicated names do not make a bad decision intelligent.

What Is a Government Algorithm?

A government algorithm is an automated process used to sort information, compare records, create alerts, calculate scores, predict behavior, or recommend an official action.

Calling every automated system “artificial intelligence” is partly a marketing trick. The term makes these products sound as if they can think, reason, and understand the consequences of their answers. They cannot. They process information according to mathematical rules and patterns created by humans.

These systems are not intelligent. In fact, they can be the opposite.

When someone uses a traditional database, such as Microsoft Access, a query run against the same unchanged records should return the same result every time. If the information in the database changes, the answer may also change. However, the database does not wake up in a creative mood and invent a customer who does not exist.

Generative artificial intelligence works differently. It uses probability to produce an answer, and the same question can generate different responses. These systems are also designed to provide an answer even when the available information is incomplete. When that happens, they can produce false information with the confidence of a man who just discovered podcasts.

The training information matters too. If the information used to train or operate a system contains bias, errors, gaps, or distorted assumptions, those problems can appear in the results. If officials continue feeding the results back into the system, one error can influence later decisions. Eventually, the government has created a monster and everyone acts surprised when it starts knocking over buildings.

Grok is an excellent example of why “artificial intelligence” should never be confused with judgment or morality. The European Commission expanded an investigation involving Grok after the system generated antisemitic material and images that included child sexual abuse material and nonconsensual intimate imagery. The United Kingdom’s communications regulator also opened a formal investigation into X over sexualized images generated through Grok. Researchers have also documented Grok producing racist and antisemitic responses. Apparently, naming something intelligent does not prevent it from behaving like the worst person in the group chat.

Agencies may describe their systems as decision support, identity verification, risk assessment, predictive analytics, fraud detection, data matching, case prioritization, anomaly detection, or eligibility screening. HA!

However, do not become trapped in an argument about whether a particular system qualifies as artificial intelligence. That question can become a convenient distraction.

The more important question is simple.

Does the system influence what the government does to a person?

If the answer is yes, the public has a right to ask how it works, how often it fails, and who is harmed when it does.

A voter removed from a registration list does not care whether the error was created by artificial intelligence, a matching formula, or a poorly maintained spreadsheet. She still has to prove that the government is wrong.

Why Accuracy Numbers Can Be Misleading

Statistics can be manipulated to make almost anything look good or bad. Government officials and technology vendors love accuracy percentages because percentages sound scientific. A system may be described as 95 percent accurate or 99 percent reliable. Those figures may be “technically” true while still concealing the serious harm that has occurred since the system was implemented.

Here is an example.

Imagine that a system reviews 100,000 people and correctly clears 99,000 of them. The agency can announce that the system is 99 percent accurate. That sounds wonderful until we look at the remaining 1,000 people.

Who are they?

Are they naturalized citizens? Are they disabled? Do they live in the same neighborhood? Do they speak the same language? Are they receiving the same public benefit? Are they members of a racial or ethnic group that is already subjected to additional government scrutiny?

An overall accuracy rate does not answer any of those questions. Instead, it can create an entirely new marginalized group that the system is, yet again, abandoning. The tech bros appear perfectly willing to say, “Fuck those people,” as long as they can continue making money from the system.

A false positive occurs when the system incorrectly identifies a person as meeting its criteria. In ordinary language, the computer accuses the wrong person.

A false negative occurs when the system fails to identify someone who actually does meet the criteria.

Both errors matter. However, false positives require special attention when a flag can lead to lost benefits, removal from voter rolls, an investigation, detention, or another government action.

You must also examine the whole picture behind every percentage, especially the denominator. An agency may report that its system helped find nine confirmed cases of fraud. That tells us almost nothing if officials do not disclose whether the system accused ten people or 10,000 people to find those nine cases.

Without the entire picture, an impressive statistic can be little more than government glitter.

Step by Step Guide

Step 1: Identify the Decision Being Made

Begin with the person affected by the system.

What happened to them?
Were they denied a benefit?
Were they removed from a voter list?
Did their application suddenly require additional review?
Were they classified as a risk?
Did an agency accuse them of providing false information?

Write the decision in one clear sentence.

For example, “The system compares voter information with federal records and sends possible noncitizen matches to local election officials.”

Do not use the agency’s promotional language. If your description contains several impressive sounding words but does not explain what happens to a person, try again.

Search the agency website for references to the program. Review meeting minutes, budgets, policy manuals, legislative testimony, presentations, procurement notices, and job advertisements.

Collect every name used for the system. Agencies sometimes change the name of a program without changing its purpose. Vendors may also use one product name in a contract and another name in marketing materials.

At this stage, you are trying to answer four questions. What does the system do? Who operates it? Who built it? Who can be harmed by it?

If the agency cannot provide clear answers, that is not proof that the system is too complicated. It may be proof that nobody wants to accept responsibility for it.

Step 2: Follow the Contract

Once you identify the system or vendor, search the appropriate government purchasing portal.

Look for the solicitation, vendor proposal, contract, statement of work, amendments, invoices, implementation schedule, and performance reports.

The statement of work may explain the system more honestly than a public announcement. It can identify the information being collected, the databases being connected, the results expected from the vendor, and the standards the system is supposed to meet.

Pay attention to subcontractors. A government agency may contract with one company while several other companies collect, store, compare, or analyze the data.

Now search the vendor’s website. Review case studies, product descriptions, conference presentations, archived pages, white papers, and sales materials.

Vendors tend to become much more descriptive when they are trying to sell something.

Compare what the vendor says with what the agency says. If the vendor describes a powerful predictive system while the agency calls it a simple administrative tool, save both descriptions.

Also look for promises about accuracy, fairness, efficiency, cost savings, and reduced employee workload. These claims give you something measurable to compare with the system’s actual performance.

Step 3: Ask Whether the System Was Tested

Before a government uses an automated system to make decisions about people, someone should determine whether it works.

That sentence should not be controversial. Yet you may discover that an agency purchased a system, connected it to public records, and began using its results without conducting a meaningful independent test.

Request all validation studies, testing plans, accuracy reports, error analyses, bias assessments, privacy reviews, civil rights reviews, quality reports, incident reports, and independent audits.

Ask for records created before the system was introduced and after it began operating. A system can perform differently when it moves from a controlled test into the confusing reality of actual human lives.

The National Institute of Standards and Technology Artificial Intelligence Risk Management Framework identifies several important characteristics of trustworthy systems. These include validity, reliability, transparency, accountability, explainability, privacy, and fairness.

The agency does not need to use this particular framework for you to use its language in a records request.

The National Institute of Standards and Technology measurement guidance also recommends examining errors, negative effects, incidents, performance limits, and whether a system functions as intended.

Ask for false positive rates, false negative rates, known limitations, acceptable error levels, and results separated by demographic or geographic category.

If officials claim the system was tested, ask who conducted the test. A vendor testing its own product is not the same as an independent evaluation.

If the agency says no testing records exist, save that response. It means officials may have placed the public inside an experiment without bothering to call it one.

Step 4: Find the Information Feeding the System

Automated systems do not make decisions from thin air. They depend on information collected somewhere else.

Request the data dictionary, record descriptions, source database list, matching rules, update schedule, retention policy, and procedures for handling missing information.

A data dictionary explains what each field means. It may show that the system uses a person’s address, age, income, language, immigration history, disability status, family structure, or other personal information.

Look for variables that can act as substitutes for protected characteristics. A system may not contain a field labeled race, but ZIP code, school, language, income, surname, or neighborhood can still produce patterns connected to race or ethnicity.

Ask how often each database is updated.

This matters because a system can correctly compare two records and still reach the wrong conclusion. One record may contain a person’s current information while another contains information that is several years old.

Citizenship changes. Names change. Addresses change. Marriages change. Disabilities change. Employment changes. Families change.

Databases are not always informed.

You should also ask what happens when information is missing. Does the system ignore the missing field? Does it send the person for manual review? Does it interpret incomplete information as suspicious?

People with unstable housing, name changes, limited access to documents, unusual family arrangements, or inconsistent historical records may be more likely to generate a mismatch. The system may call this an anomaly. The person living through it may call it Tuesday.

Step 5: Count the People Affected

Request aggregated records showing how many people were screened, flagged, investigated, denied, removed, cleared, referred, reinstated, or approved.

Ask for the figures by month or quarter. A yearly total can hide a sudden increase in errors after the system was updated or connected to a new source of information.

Now ask for appeals and reversals.

How many people challenged the decision? How many succeeded? How long did the process take? Were benefits, voting rights, services, or opportunities withheld while the person waited?

A reversal rate can reveal problems that the original accuracy report did not capture.

Imagine that an agency denies 500 applications and later reverses 300 of those decisions. The agency may describe the appeals process as successful. I would describe the original process as wrong an alarming amount of the time.

Request complaint summaries, correction requests, help desk tickets, employee concerns, known issue reports, system outages, and communications discussing inaccurate results.

Useful search terms include false match, incorrect flag, duplicate record, data quality, manual override, appeal, correction, exception, and unintended impact.

Do not request names or other identifying information when aggregated records will answer the question. People should not lose more privacy because the government mishandled their information.

Step 6: Compare the Error Rates

Raw totals can create a misleading picture. If one group contains more people, it may naturally produce more flags. You need rates.

Calculate the flag rate by dividing the number of people flagged in a group by the total number of people from that group who were screened.

Then calculate the reversal rate by dividing the number of corrected decisions by the total number of adverse decisions for that group.

Compare the results by race, ethnicity, age, sex, disability, language, citizenship history, county, neighborhood, income level, or benefit category when that information is legally available in aggregated form.

For example, if 100 people from one group were screened and 20 were flagged, the flag rate is 20 percent. If 1,000 people from another group were screened and 50 were flagged, the second group produced more total flags but had a much lower flag rate of 5 percent.

This is why the denominator matters.

A difference between groups does not automatically prove intentional discrimination. It does show where further investigation is required.

The cause may be poor data, an outdated source, a matching rule, a documentation requirement, an inaccessible appeal process, or the way human reviewers interpret the system’s results.

Document every calculation. Another person should be able to reproduce your work. If an agency challenges your findings, you want to answer with the numbers, not a debate about feelings.

Step 7: Examine the Human Review

When confronted about an automated decision, agencies often say that a human makes the final determination.

That statement is supposed to make everyone feel better.

It should not, at least not until we know what the human reviewer actually does.

Request the reviewer instructions, training materials, productivity requirements, time limits, override procedures, escalation rules, and quality reviews.

Ask how often employees accept the automated recommendation and how often they reject it.

If a reviewer processes hundreds of cases, has only a few minutes for each one, and must explain every disagreement with the computer, the promised human review may not be meaningful.

People tend to trust automated results, especially when the system appears technical or authoritative. A reviewer may assume the computer has access to information that she does not understand. She may also fear being blamed if she overrides the recommendation and something later goes wrong.

Find out whether the affected person is told that an automated system influenced the decision.

Read the notice sent to them. Does it explain what information was used? Does it identify the reason for the decision? Does it provide a meaningful correction process? Does it explain the right to appeal?

A person cannot challenge an error she cannot see.

Step 8: Build a File for Every Confirmed Error

Create a separate folder for each verified error.

Include the original notice, the stated reason, the records used, the person’s correction documents, the appeal, the final outcome, and the amount of time required to fix the problem.

Remove personal information before sharing the case publicly unless the affected person has given informed permission.

Now compare the cases.

Are people with name changes repeatedly affected? Are naturalized citizens appearing in the errors? Are records from one county outdated? Are disabled applicants being forced into categories that do not describe their situations? Are certain groups required to produce more documentation than others?

One error can be described as an unfortunate mistake. Ten similar errors begin to show a pattern.

Do not focus only on whether the final decision was corrected. Examine what happened while the person waited.

Did she miss an election? Did he lose food assistance? Did a family fall behind on rent? Did someone have to hire an attorney to prove a fact that the government already possessed?

An error rate does not measure the emotional, financial, and practical cost of being wrongly accused by your own government.

Step 9: Compare the Promises With the Results

Create a chart with the agency’s promises on one side and your findings on the other.

If the vendor promised high accuracy, compare that promise with the recorded errors and reversals.

If officials promised meaningful human review, compare that promise with employee instructions and override rates.

If the agency promised current information, compare that promise with cases caused by outdated records.

If officials claimed the system would reduce bias, ask whether they measured bias before or after introducing it.

Use careful language in your final conclusions. Say the records show, the available data indicates, or the results raise questions.

Do not claim intentional discrimination unless the evidence supports that conclusion. A system can create discriminatory results without anyone writing a rule that openly targets a protected group.

Precision makes your investigation stronger. It also prevents an agency from using one exaggerated sentence to avoid answering the rest of your findings.

Step 10: Demand a Measurable Remedy

Do not finish your investigation by asking officials to do better. “Do better” is not a policy. It is a decorative suggestion.

Ask for a specific action.

The agency may need to suspend the system, conduct an independent audit, publish error rates, review previous decisions, notify affected people, improve appeal notices, correct source records, restrict data sharing, or require continuing civil rights testing.

Send your findings to the agency inspector general, civil rights office, legislative oversight committee, elected officials, public interest attorneys, and independent journalists covering the issue.

Lead with the harm. Explain the evidence. Include the calculations. Attach the supporting records. Identify the questions the agency did not answer.

Government officials may claim that the public does not understand the technology. Do not allow that argument to end the conversation.

You do not need to understand every line of computer code to prove that eligible voters were flagged, qualified applicants were denied, or hundreds of decisions were later reversed.

You understand the outcome perfectly well.

Public Records Request Language

Under the applicable public records law, I request all records concerning the design, purchase, testing, deployment, operation, review, and evaluation of the automated system known as [system name].

Yes, I am asking for everything connected to this system. If the government is going to allow a computer to make decisions about people, the public deserves to know who built it, how it works, and what happens when it gets something wrong.

This request includes contracts, statements of work, vendor proposals, validation studies, testing results, accuracy reports, bias assessments, privacy reviews, civil rights reviews, data dictionaries, source database lists, matching rules, reviewer instructions, training materials, error reports, complaint records, appeal outcomes, override records, performance reports, audit logs, and communications concerning false matches or inaccurate results.

I also request aggregated records showing the number of people screened, flagged, cleared, investigated, denied, removed, referred, appealed, reinstated, or otherwise affected by the system. Where these records are maintained, please provide the figures by month and by every available demographic or geographic category.

I know this is a lot. That is the point. You cannot audit one tiny piece of a system and pretend you understand what the entire machine is doing. I promise this process gets easier once you learn what to request and where the government likes to hide the useful parts.

I am not requesting personally identifying information. Please provide every reasonably separable portion of any record containing exempt information. If any record or portion of a record is withheld, please identify the specific legal basis for withholding it and describe the withheld material sufficiently to permit review of that decision.

In ordinary language, do not deny the entire request because one part of one document may be exempt. Remove the legally protected information and give me the rest.

Make sure to thank many of our independent journalists who have modeled the best ways to request these records through their trial and error!!

Warning Signs That Require More Investigation

Pay attention when an agency refuses to identify the vendor, produces only one broad accuracy number, has no independent validation study, does not track reversals, or claims that demographic testing is impossible. This is not just a red flag, but the entire color guard telling us there is something fishy going on.

Some other things that should make you feel ookie include outdated source databases, sudden increases in flags, high appeal success, employees who rarely override recommendations, and notices that fail to explain why a person was targeted.

You should also become very suspicious when an agency says the system cannot be examined because it belongs to a private company. The vendor may own the software. The government still owns the decision.

Officials cannot outsource public power and then pretend they also outsourced responsibility. Which is one of the goals of this administration.

How To Report Your Findings Responsibly

Separate documented facts from your conclusions. Explain which records you reviewed, what period they covered, what information was missing, and how you calculated each rate. Show your receipts. Then explain your research process and conclusions in language that an ordinary person can understand without needing three advanced degrees and a government issued decoder ring.

Protect the people affected by the system. Never publish identifying information simply because it appeared in a public record. Ask permission before sharing someone’s story and explain exactly how that information will be used. Consent is always required. It is one of the things that separates us from the fascists.

Give the agency and vendor an opportunity to respond. Include their explanations accurately. Keep emotion and opinion out of your reported findings. Otherwise, your work may sound more like complaining than journalism, and they will use that as an excuse to ignore the evidence.

You do not have to agree with their response. You do have to represent it honestly.

Responsible reporting does not mean soft reporting. It means building a case so solid that it can survive the denials, excuses, public relations statements, and anger of the people exposed by it.

Show the receipts. Protect the people. Report the truth.

RAWR!

In Conclusion

An algorithm does not suddenly become neutral just because it lives inside a government computer or is owned by some contractor who charged taxpayers millions of dollars for it. It is still built by people. People decide what information matters, what patterns look suspicious, how many errors are acceptable, and which poor bastard will be forced to prove that the computer is wrong.

When these decisions are hidden, the burden never falls on the people who built the system or the government officials who approved it. It falls on the person with the least amount of power.

She may not know which database accused her. She may not know which rule was applied, whether a human being ever looked at the result, or how many other people received the same terrifying letter.

Then the agency tells her to appeal.

Of course it does.

This is not efficiency. The government created an obstacle, dropped it directly into someone’s life, and then congratulated itself for providing a form she can use to beg them to remove it. Sigh. Eye roll. Rage scream.

We have to stop treating automated decisions as if they came down from some perfect digital authority in the sky. Computers do not eliminate human bias. They can hide it, repeat it, and apply it to thousands of people before anyone notices the pattern. By then, the people responsible are usually standing behind the computer, shrugging and blaming “the system.”

Find the contract. Follow the information. Ask for the testing. Count the errors. Examine the appeals. Compare who gets flagged and who gets cleared. Find out what the alleged human reviewer actually does.

Then ask the most important question.

Who knew the system was failing, and why were the people being harmed expected to discover it one error at a time?

The computer did not make this choice by itself.

People did.

Find them. Name them. Show the receipts. Out what they did.

Sources

Kitty’s Resistance Projects

Support Resistance Kitty’s Work

R – We Tread On Tyrants Light Style Unisex Long Sleeve Tee