RSG #308: How To Reconstruct A Government Algorithmic Decision System

Resistance Survival Guide #308

Government agencies increasingly use automated systems to decide who receives assistance, who is investigated, where police officers are deployed, which families receive additional scrutiny, and which applications are rejected. These systems may be described as artificial intelligence, predictive analytics, risk assessment, decision support, fraud detection, resource allocation, prioritization, or case management.

The name matters less than the function. When software assigns a score, recommends an action, identifies a supposed risk, or changes how a person is treated, it has become part of the government decision making system.

Reconstructing a government algorithmic decision system means identifying the data that enters the system, the rules that process it, the score or recommendation it produces, and the human actions that follow. You do not need to be a computer scientist. You need patience, organized records, carefully written public records requests, and a willingness to trace the entire decision process.

Why Government Algorithms Require Public Scrutiny

Government algorithms are often presented as neutral tools that make agencies faster and more consistent. However, every system contains human decisions. Someone chooses the goal, the data, the variables, the thresholds, the definitions of success, and the consequences attached to each result.

A system can be mathematically accurate while still measuring the wrong thing. It can also reproduce existing discrimination because it learns from records created by unequal institutions. A policing system trained on arrest data may learn where police historically made arrests rather than where crime actually occurred. A child welfare system may treat previous contact with public assistance programs as evidence of family risk. A fraud detection system may punish ordinary reporting mistakes as intentional deception.

The National Institute of Standards and Technology identifies validity, reliability, transparency, explainability, privacy, accountability, and the management of harmful bias as essential elements of trustworthy artificial intelligence. The Government Accountability Office organizes government AI accountability around governance, data, performance, and continuous monitoring. These standards provide investigators with a useful framework for deciding which records to request.

A Government Algorithm Is More Than Its Source Code

Investigators often begin by asking for source code. That can be useful, but source code is only one part of the system. Agencies and vendors may also claim that code is protected by trade secret, cybersecurity, or licensing restrictions.

The more effective approach is to investigate the entire decision system. This includes contracts, data sources, policy manuals, scoring rules, validation studies, staff instructions, user interfaces, appeal notices, override procedures, error reports, and communications between the agency and its vendor.

A complete reconstruction should answer five questions. What information enters the system? What rules or statistical methods process that information? What result does the system produce? How do government employees use that result? What happens to the person affected by the decision?

The AI Now Institute has emphasized that reviewing an automated system requires more than studying its mathematics. Investigators must examine how workers, administrators, policies, and institutional practices turn an algorithmic output into a real government action.

Real Cases Show Why Reconstruction Matters

Michigan used an automated system known as MiDAS to identify supposed unemployment fraud. The system generated thousands of incorrect fraud findings and exposed residents to penalties, collection actions, and severe financial harm. A review of approximately 22,000 automated determinations found that 93 percent did not involve fraud. The system demonstrates why investigators must obtain error rates, appeal outcomes, collection records, and evidence showing whether humans meaningfully reviewed the results.

Idaho used a statistical budget tool to calculate Medicaid assistance for people with intellectual and developmental disabilities. Litigation revealed serious problems with the system, the notices sent to recipients, and the ability of affected people to understand how their budgets had been calculated. A federal court found that the unreliable and arbitrary nature of the tool violated due process. The case demonstrates why investigators should request individual calculation records, manuals, formulas, notices, audit results, and appeal procedures.

Allegheny County, Pennsylvania, uses a predictive system to assist workers screening child welfare reports. The system processes numerous government data elements and produces a family screening score. The controversy surrounding the tool shows why investigators must identify the outcome being predicted, the records used as variables, the population used to validate the system, and the role workers play after receiving a score.

Step by Step Guide

Step 1: Define The Government Decision You Are Investigating

Begin with the decision rather than the technology. Write one sentence describing what the government is deciding. Examples include whether a person receives Medicaid services, whether an unemployment claim is labeled fraudulent, whether a family is investigated, whether a housing applicant receives priority, or whether police resources are sent to a neighborhood.

Next, identify the agency, department, office, and program responsible for that decision. Record the names of senior officials, program administrators, technology officers, procurement employees, records custodians, and legal counsel. This creates the first layer of your investigation map.

Do not assume the agency publicly calls the system an algorithm. Search for terms such as risk score, decision support, predictive model, analytics platform, resource allocation model, fraud detection, eligibility engine, prioritization tool, scoring matrix, automated review, case management system, and artificial intelligence.

Step 2: Find The System Before Filing Records Requests

Search the agency website, meeting agendas, budget documents, procurement portals, annual reports, technology plans, job advertisements, grant applications, legislative testimony, privacy notices, and vendor announcements.

A job advertisement seeking someone to manage predictive analytics may reveal a system that never appears on the agency website. A budget amendment may reveal a vendor payment. A governing board agenda may identify the contract number. A grant application may explain the project more honestly than a public relations announcement.

Search the agency name alongside terms such as algorithm, analytics, scoring, model, risk, automation, fraud, artificial intelligence, modernization, decision support, machine learning, and vendor.

Also search the agency website for common procurement file types, including PDF files, spreadsheets, presentation files, and contract attachments.

Step 3: Build A Preliminary System Map

Create a working document with five columns labeled input, process, output, human action, and consequence.

Under input, list every known or suspected data source. This may include criminal records, benefit history, medical claims, school attendance, employment records, housing information, location data, financial records, previous investigations, demographic information, or information purchased from private companies.

Under process, record what the system appears to calculate. It may apply fixed rules, compare records, estimate risk, identify patterns, rank people, flag inconsistencies, or predict a future event.

Under output, record whether the system produces a score, category, alert, recommendation, ranking, denial, payment amount, investigation referral, or enforcement action.

Under human action, identify who sees the result and what that person is instructed to do.

Under consequence, record what happens to the affected person. This is the most important column because an algorithmic score has little meaning until it changes a government action.

Step 4: Trace The Procurement History

Search for requests for proposals, invitations to bid, vendor responses, evaluation materials, contracts, purchase orders, invoices, amendments, renewal notices, implementation plans, and statements of work.

The original request for proposals may describe the agency problem, the desired system, the required data connections, the expected accuracy, and the planned consequences. The winning proposal may identify subcontractors, technical components, software products, data brokers, and previous government clients.

Contract amendments are especially important. An agency may begin with a limited pilot and later expand the system into additional programs without significant public discussion.

Record every contract number, payment amount, vendor name, subcontractor, project manager, start date, renewal date, and termination clause. Search each company through state business registries, lobbying databases, campaign finance records, and government contract databases.

Step 5: File Several Focused Public Records Requests

Do not place every possible record into one enormous request. Agencies can delay or reject an overly broad request more easily. Divide the investigation into focused requests for procurement, technical design, data, performance, communications, training, and appeals.

MuckRock maintains examples of public records requests concerning government algorithms. Its requests commonly seek code, policies, procurement records, financial documents, implementation materials, and records describing how a system is used.

Always request records in their original electronic format. A spreadsheet converted into a PDF may lose formulas, hidden fields, metadata, filters, and other valuable evidence.

Ask the agency to provide nonexempt portions of partially exempt records. Also ask it to identify the legal basis for every withholding and to provide an index describing withheld records when required by the applicable law.

Step 6: Request The Technical Design Records

Request system architecture diagrams, data flow diagrams, specifications, decision rules, scoring formulas, model documentation, variable lists, thresholds, weights, source code, pseudocode, configuration files, model cards, technical manuals, user manuals, system requirement documents, change logs, and version histories.

If the agency denies access to source code, narrow the request rather than abandoning it. Ask for plain language explanations, scoring tables, variable weights, threshold settings, data dictionaries, user interface screenshots, validation reports, manuals, and records explaining how the output is calculated.

A vendor may own the software, but the government still possesses records showing how public officials use it. Procurement documents, training manuals, configuration records, and decision notices can reveal much of the system even when the code remains unavailable.

Step 7: Identify Every Data Source

Request a complete list of databases, datasets, fields, variables, tables, and outside sources accessed by the system. Ask for data dictionaries that define each field.

A variable name can be misleading. A field labeled stability might actually measure the number of times a person changed addresses. A field labeled engagement might measure missed appointments. A field labeled risk history might combine police contact, public benefits, medical information, and previous allegations.

Request records showing where each data element originates, how often it is updated, how errors are corrected, how long it is retained, and whether individuals can view or challenge it.

Do not request personally identifiable records belonging to vulnerable individuals. Ask for deidentified data, aggregate reports, schemas, data dictionaries, blank forms, and representative test records.

Step 8: Determine What The System Is Actually Predicting

Government officials may say a system predicts danger, fraud, neglect, instability, or need. Those terms can hide the actual statistical target.

Request records defining the predicted outcome. Ask whether the system predicts an actual harmful event or simply predicts another government action, such as an arrest, investigation, benefit termination, or child removal.

This distinction matters. A system trained to predict future arrests may be measuring patterns of police enforcement rather than criminal conduct. A system trained to predict child removal may reproduce previous agency practices rather than identify actual harm.

Request the precise outcome definition, observation period, training population, comparison groups, exclusion rules, and criteria used to decide whether a prediction was correct.

Step 9: Obtain Validation And Accuracy Records

Request every validation study, accuracy report, audit, quality assurance review, performance dashboard, error analysis, pilot evaluation, testing report, and independent assessment.

Ask for false positive rates, false negative rates, precision, recall, sensitivity, specificity, calibration results, missing data rates, and performance across demographic groups. You do not need to calculate these measures immediately. Obtaining the records is the first goal.

False positives occur when the system identifies a problem that is not present. False negatives occur when the system fails to identify a real problem. The consequences of these errors are not equal. A false fraud accusation can trigger debt collection. A false child welfare risk score can expose a family to investigation. A false negative in a public safety system may leave a genuine threat unaddressed.

The NIST AI Risk Management Framework Playbook recommends defining acceptable performance limits and tracking errors, incidents, and negative impacts. Those recommendations provide useful language for public records requests.

Step 10: Investigate Human Review And Overrides

The presence of a government employee does not automatically create meaningful human review. Workers may be expected to follow the system, may not understand how it works, or may fear consequences for overriding its recommendations.

Request staff policies, training materials, override instructions, supervisor guidance, user permissions, audit logs, and records showing how frequently workers accept or reject the system output.

Ask whether employees must provide a reason when they override a score. Also ask whether supervisors review overrides and whether employees are evaluated based on agreement with the system.

Compare the agency claim that humans make the final decision with the actual override data. A system followed in nearly every case may function as the real decision maker even when a human formally approves the result.

Step 11: Reconstruct The Notice And Appeal Process

Collect blank copies of every notice sent to people affected by the system. Request denial letters, benefit calculation notices, fraud notices, investigation referrals, appeal forms, hearing instructions, and explanation templates.

A meaningful notice should tell a person what decision was made, which facts were used, how the decision was calculated, who made it, and how to challenge an error.

Request records showing appeal numbers, appeal outcomes, reversal rates, processing times, reasons for reversal, and the number of people who never completed an appeal.

A high reversal rate can reveal a defective system. A low appeal rate does not prove accuracy. It may show that notices were confusing, deadlines were short, legal assistance was unavailable, or affected people did not understand that an automated system had influenced the decision.

Step 12: Request Complaints, Incidents, And Internal Warnings

Request complaints from residents, advocacy organizations, employees, contractors, inspectors, auditors, and elected officials. Ask for help desk tickets, incident reports, error logs, bug reports, corrective action plans, meeting notes, and communications discussing unexpected outcomes.

Search for phrases such as inaccurate result, incorrect score, system error, false positive, data mismatch, duplicate record, appeal reversal, manual correction, unintended consequence, bias, complaint, outage, and urgent fix.

Internal communications may show that employees recognized a problem long before the agency publicly acknowledged it. Record when each warning appeared, who received it, and what action followed.

Step 13: Compare Policy With Actual Practice

Read the official policy first. Then compare it with training materials, screenshots, logs, employee communications, appeal files, and testimony from affected people.

A policy may state that a score is advisory while training materials instruct workers to follow it. An agency may claim that protected information is excluded even though the data dictionary contains closely related proxy variables. A contract may require annual testing while invoices show no independent audit was purchased.

Create a contradiction log. For every public claim, record the document that supports or challenges it. This makes the final investigation easier to verify.

Step 14: Test The System With Public Evidence

You may not receive enough information to reproduce the software exactly. However, you can still reconstruct its operational logic.

Collect blank assessment forms, scoring charts, notices, manuals, screenshots, sample cases, appeal decisions, court filings, and testimony. Use these records to determine which facts increase or decrease a score.

Build hypothetical cases using fictional people. Change one variable at a time and record how the result changes. Never submit false information to a government agency or access a restricted system. Testing should rely on public records, approved demonstrations, open data, or authorized access.

When possible, ask an independent statistician, data journalist, civil rights attorney, social scientist, or subject matter expert to review your findings.

Step 15: Build A Decision Timeline

Create a timeline beginning with the agency discussion that led to the system. Include the procurement process, vendor selection, testing, pilot launch, full deployment, major updates, complaints, audits, lawsuits, and public statements.

Then create a second timeline for the decision itself. Show when data enters the system, when the score is generated, when an employee sees it, when the government acts, when the person receives notice, and when an appeal becomes available.

These timelines expose delays, hidden decision points, and contradictions. They can also reveal whether the agency continued using a defective system after learning about serious errors.

Step 16: Challenge Improper Withholdings

When an agency claims trade secret protection, ask it to explain why each withheld record qualifies and whether the government promised confidentiality during procurement.

Request a redacted version. Ask the agency to separate protected commercial details from records explaining government policy, system outputs, validation results, and public consequences.

Appeal denials within the required deadline. Public records laws vary by jurisdiction, so consult the relevant state freedom of information organization, local civil liberties organization, legal aid office, or public records attorney.

Do not allow a vendor contract to become the final word on public access. A private company should not be able to eliminate public oversight simply by placing government decision rules inside proprietary software.

Step 17: Publish The Evidence Responsibly

Organize your evidence into procurement, design, data, performance, human review, consequences, appeals, and communications.

Upload original records to a stable public archive such as DocumentCloud when legally and ethically appropriate. Remove personal information belonging to benefit recipients, children, patients, witnesses, and other vulnerable people.

Clearly distinguish facts, agency claims, vendor claims, expert analysis, and your own conclusions. Publish the records that support each major statement.

Explain what remains unknown. An honest gap in the evidence is more credible than speculation presented as certainty.

Public Records Request Template

Pursuant to the applicable public records law, I request records concerning any automated, algorithmic, predictive, statistical, artificial intelligence, machine learning, scoring, risk assessment, decision support, prioritization, eligibility, fraud detection, or resource allocation system used by this agency in connection with the identified program.

Please provide contracts, proposals, statements of work, purchase orders, invoices, amendments, renewal records, vendor communications, implementation plans, system architecture records, data flow diagrams, technical specifications, variable lists, data dictionaries, scoring rules, threshold settings, formulas, model documentation, validation reports, accuracy reports, bias assessments, audits, quality assurance records, training materials, user manuals, screenshots, policies, staff instructions, override procedures, appeal records, error logs, complaint records, incident reports, change logs, and version histories.

Please provide records in their original electronic format whenever possible. For spreadsheets, databases, or structured data, please provide the native file rather than a converted PDF.

This request does not seek personal information identifying individual program participants. Aggregate or deidentified records are acceptable where necessary to protect privacy.

If any portion of a record is exempt, please provide all reasonably separable nonexempt material. Please identify the specific legal authority supporting each withholding and describe the records being withheld.

Warning Signs That Deserve Further Investigation

A government agency should receive additional scrutiny when it cannot identify the system owner, cannot explain the score, has no documented accuracy threshold, does not track errors, refuses to disclose validation results, relies entirely on a vendor, or provides no meaningful way for affected people to challenge a decision.

Other warning signs include using historical government actions as proof of actual risk, testing the system only on the same data used to create it, changing the model without public notice, failing to preserve earlier versions, and claiming that human review exists without producing override data.

The strongest warning may be an agency that does not know how its own system works. Public officials remain responsible for government decisions even when a contractor designed the software.

The Goal Is Accountability, Not Technical Theater

An investigation does not succeed merely because it uncovers a complicated formula. The goal is to determine whether the government system is lawful, accurate, fair, understandable, and open to challenge.

Follow the decision from beginning to end. Identify the policy goal. Trace the data. Examine the formula. Measure the errors. Document human involvement. Study the consequences. Review the appeal process. Then compare the system the agency describes with the system people actually experience.

Government algorithms should not receive less scrutiny than human officials. When software helps decide who receives help, who loses resources, or who becomes a target, the public has a right to understand the machinery of that decision.

In Closing

Government algorithmic decision systems often hide behind technical language, vendor secrecy, and fragmented records. However, every system leaves a paper trail. Contracts reveal why it was purchased. Data dictionaries reveal what it measures. manuals reveal how employees use it. Error reports reveal where it fails. Appeals reveal who pays the price.

The public does not need permission to ask how government power is being exercised. By reconstructing the data, rules, people, and consequences surrounding an automated system, investigators can turn an invisible decision process into evidence that communities, journalists, attorneys, and elected officials can examine.

Source List

Kitty’s Resistance Projects

Support Resistance Kitty’s Work

R – We Tread On Tyrants Light Style Unisex Long Sleeve Tee