Government case studyControlled evaluation planRedacted

Auditable AI for High-Stakes Compliance Work

A redacted prior-team-experience case study covering citation-backed research, sensitive-data guardrails, audit evidence, and reviewer-gated control-workbook stewardship.

See the results
3 problems
  1. 01 Authorize and prepare
  2. 02 Baseline and exercise
  3. 03 Red-team and decide
Redacted federal compliance organization
Client
Redacted federal compliance organization
Industry
Federal cybersecurity and regulatory compliance
Implementation
8-week controlled proof-of-concept plan
Scope
Redacted / Federal / Compliance / RAG / Human Review

01 Original problem

The operational friction before the build.

Before the build, Galang AI found 3 problems at Redacted federal compliance organization.

  1. 01

    Analysts had to research a large, versioned body of policy, technical guidance, and control workbooks under time pressure.

  2. 02

    Any AI assistance had to prevent sensitive identifiers from reaching a model, preserve source provenance, and produce evidence suitable for reviewer reconstruction.

  3. 03

    Automation could not replace authoritative policy decisions, compliance determinations, or release approval.

Redacted federal compliance architecture with protected data, cited retrieval, and human review

02 Exact deliverables

What was actually designed and delivered.

Galang AI delivered 4 pieces of work for Redacted federal compliance organization: citation-backed compliance assistant, pre-inference sensitive-data guardrail, reviewer-gated workbook stewardship and security and accountability controls.

  1. 01

    Citation-backed compliance assistant

    Hybrid retrieval over approved policy and technical sources, with source context, citations, and audit metadata attached to responses.

  2. 02

    Pre-inference sensitive-data guardrail

    Server-side screening that blocks high-confidence sensitive patterns before model calls and creates incident and audit records.

  3. 03

    Reviewer-gated workbook stewardship

    Pinned source manifests, version and hash provenance, proposed control-language changes, approve/reject/edit/undo actions, and candidate workbook export.

  4. 04

    Security and accountability controls

    MFA, role-based access, organization-scoped permissions, audit drilldowns, concurrency controls, and explicit separation between candidate drafts and official releases.

03 Architecture

Protected, reviewer-gated evidence flow

Sensitive input is stopped before inference; permitted questions retrieve from pinned sources; every draft remains attributable and reviewable.

  1. 01

    Authorized user and MFA

  2. 02

    Pre-inference sensitive-data screen

  3. 03

    Hybrid retrieval over pinned sources

  4. 04

    Cited draft plus audit metadata

  5. 05

    Human review

  6. 06

    Candidate export and separate release approval

04 Implementation timeline

8-week controlled proof-of-concept plan

The current evaluation plan is a controlled PoC using synthetic or sanitized data. It is not represented as an agency-wide production rollout.

  1. 01–02

    Authorize and prepare

    Lock data boundaries, success metrics, source inventory, roles, and the pilot environment.

  2. 03–05

    Baseline and exercise

    Measure manual workflows, train pilot users, and run citation, guardrail, and workbook scenarios.

  3. 06–08

    Red-team and decide

    Run security tests, analyze evidence, and issue a go, conditional-go, or no-go recommendation.

05 Before and after

Measurements with evidence labels.

Measured facts, observed workflow changes, evaluation targets, and missing baselines are kept distinct.

  1. Research time

    Evaluation target

    Before

    Manual baseline to be measured in the PoC

    After

    Target: at least 60% median reduction in draft-response time

  2. Citation precision

    Evaluation target

    Before

    Baseline to be established on a locked question set

    After

    Target: at least 90% precision and zero accepted fabricated citations

  3. Sensitive-data blocking

    Evaluation target

    Before

    Manual policy and user discipline

    After

    Target: 100% of high-confidence SSN/EIN test cases blocked before inference

  4. Control-workbook coverage

    Observed

    Before

    Distributed manual research across many workbooks

    After

    58 current workbooks pinned in the evaluation corpus

06 Work samples

See the work, not just the summary.

Redacted federal compliance agent demonstration

07 Client voice

Client testimony

“James Galang's involvement and innovation in the review process has been very impressive. I wish I had the compliance tool he created sooner.”
Division CISOFederal leader • identity and organization protected

Methodology

How this case was substantiated

  • Uses a locked golden question set, negative set, guardrail red-team prompts, reviewer rubrics, time-on-task measurement, and source-provenance checks.
  • Separates deterministic evidence, AI recommendations, unresolved source gaps, and human decisions in the audit record.
  • Uses synthetic or sanitized data for the PoC; production sensitive data is out of scope.

Limitations

What the evidence does not prove

  • The customer, agency, system name, screenshots, URLs, internal identifiers, source versions, and operational data are intentionally omitted or generalized.
  • Targets are evaluation thresholds, not claimed achieved outcomes.
  • The system does not certify compliance, publish an official control workbook, or replace authorized reviewers.
  • Licensed benchmark material and agency data are not reproduced in this public case study.

Map the operational friction before choosing the technology.

We can scope the workflow, evidence, adoption plan, security boundaries, and success measurements before recommending a build.

All case studies