# Federal Data Field Guide: LLM & Agent Operating Instructions

This repository contains the authoritative source code, raw data, and structural definitions for the **Federal Data Field Guide** (April 2026, V.1), produced by the Executive Fellowship in Applied Technology Policy (a joint program of UC Berkeley's School of Information and Goldman School of Public Policy). When an LLM or AI agent is invoked to assist with queries using this guide, it MUST strictly adhere to the guidelines, schemas, and operational constraints defined below.

---

## I. Core Mission & Scope Constraints

1. **Scope & Exclusions**: This guide focuses on publicly available datasets created, maintained, and published by executive branch agencies of the U.S. federal government. It **excludes** sensitive or classified datasets, as well as derivative works such as reports or interactive web tools that use data.
2. **Verifiable Information Only**: Never invent, extrapolate, or hallucinate non-existent federal agencies, datasets, or classification codes.
   * For datasets explicitly included in the Field Guide, verify details against the guide's metadata.
   * For datasets **not** listed in the Field Guide, verify their validity against official federal sources, such as **Data.gov** or primary federal agency websites (`.gov` / `.mil`). 
   * *Note*: Data.gov is the official catalog but is not exhaustive; omission from Data.gov does not invalidate a dataset if it is published on an official agency website.
3. **Ontology Flexibility & Extension**: Agents can and should use the Field Guide's 8-part category ontology to classify **any** valid federal dataset, whether it is cataloged in the Field Guide or discovered via external federal sources.
4. **Platypus Principle (Multi-Category Classification)**: Datasets do not always fit neatly into a single category (hence the duck-billed platypus mascot). When a dataset exhibits multi-category traits, assign a **Primary Category** and list relevant **Secondary Categories**.
5. **Direct & Objective Tone**: Communicate plainly and directly. Avoid hyperbole, marketing fluff, or conversational setup sentences. Begin responses immediately with structured tables or direct facts.

---

## II. Primary Data Categories & Functional Ontology

All federal datasets—whether listed in the Field Guide or externally discovered—must be categorized using the **eight primary categories** defined in the Federal Data Field Guide. Agents must not use external domain groupings (e.g., "Health", "Economy", "Environment"), as the guide's taxonomy is organized by **collection methods, policy frameworks, and functional use cases**.

| Category Identifier | Category Name | Primary Definition & Scope | Primary Collection Modes | Key Governing Laws & Policies |
| :--- | :--- | :--- | :--- | :--- |
| `CAT-STAT` | **Statistical Data** | Measures group- and population-level characteristics of people, businesses, and the economy to inform planning, policymaking, and resource allocation. | Surveys, interviews (in-person, phone, mail, web), third-party reports (e.g., law enforcement), administrative supplements. | CIPSEA, Title 13 (Census), Title 26 (IRS Sec. 6103), PRA, Privacy Act, FERPA, HIPAA. |
| `CAT-ADMIN` | **Administrative Data** | Generated through routine government operations and transactions, such as program eligibility, benefit delivery, regulation, and tax administration. | Program application forms, regulatory filings, electronic records, internal agency databases, third-party institutions (lenders, states). | Privacy Act (SORNs), FOIA, OPEN Government Data Act, HMDA, NEPA, HIPAA, FERPA. |
| `CAT-GEO` | **Geospatial Data** | Contains specific location attributes (latitude/longitude, addresses, or spatial references) capturing spatial and environmental information. | Satellite and aerial imagery, environmental sensors, GPS/address surveys, interagency or commercial data agreements. | Geospatial Data Act (GDA), National Spatial Data Infrastructure (NSDI), Privacy Act, GeoPlatform. |
| `CAT-SCI` | **Scientific Data** | Generates, validates, and replicates findings that advance knowledge across scientific disciplines, clinical trials, and theoretical models. | Research sensors, biological sampling, time-limited field measurements, spaceflight experiments, continuous monitoring. | FAIR Principles, Evidence Act, Information Quality Act (IQA), OSTP Public Access Policies, Common Rule. |
| `CAT-ACCT` | **Accountability Data** | Measures government functions, spending, policy implementation, and service delivery to provide transparency into government activities. | Administrative financial systems, operational software, performance metrics, compliance logs, FOIA request logs. | DATA Act, FFATA, FOIA, Food and Nutrition Act, Information Quality Act (IQA), OPEN Government Data Act. |
| `CAT-EVAL` | **Evaluation Data** | Analyzes the design, implementation, and impact of government initiatives to determine value, relevance, and areas for program improvement. | Mixed methods (one-time or targeted studies), surveys, qualitative interviews, experiments, secondary administrative/statistical data. | Foundations for Evidence-Based Policymaking Act (Evidence Act - Agency Learning Agendas), Privacy Act (SORNs), CIPSEA, IQA. |
| `CAT-NAV` | **Navigation Data** | Characterizes federal and non-federal resources in a structured way to help citizens and software identify relevant benefits, services, or information. | Centralized resource inventories, administrative/reference lists, interagency portals, API-driven web catalogs. | OPEN Government Data Act (open by default/machine-readable), IQA, FDAAA, Uniform Guidance, FFATA. |
| `CAT-REF` | **Reference Data** | Provides structure, standardization, and consistency (classification codes, boundaries, official names) across other data sources to enable interoperability. | Administrative coding processes, statistical geography definitions, legislative/regulatory directives, geospatial surveying. | Evidence Act Data Inventories, OPEN Government Data Act, Geospatial Data Act (FGDC standards), IQA. |

---

## III. Legal & Governance Framework Ontology

When describing the legal context, privacy constraints, or access restrictions of any dataset, reference the appropriate statutory or policy frameworks below:

* **Privacy & Security**:
  * **Privacy Act of 1974**: Governs PII handling; requires Systems of Records Notices (SORNs) and limits disclosures to routine uses.
  * **FISMA / OMB Circular A-130**: Governs cybersecurity, risk management, and federal IT as strategic assets.
  * **CIPSEA (Title III of Evidence Act)**: Protects data collected under a pledge of confidentiality for statistical/evidence-building purposes only; shields data from enforcement, tax, or immigration uses.
  * **Title 13 (Census) & Title 26 (IRS Sec. 6103)**: Enforces strict confidentiality for census responses and tax return information.
  * **HIPAA & FERPA**: Sector-specific privacy laws protecting health records and student educational records.
* **Transparency & Evidence-Building**:
  * **OPEN Government Data Act (OGDA)**: Mandates that non-sensitive data be "open by default" in machine-readable formats and indexed on Data.gov.
  * **Evidence Act of 2018**: Requires agency Learning Agendas, Comprehensive Data Inventories, and establishes Chief Data Officers, Evaluation Officers, and Statistical Officials.
  * **FOIA**: Mechanism for public record requests, subject to statutory privacy exemptions.
* **Data Quality & Governance**:
  * **Paperwork Reduction Act (PRA)**: Requires OMB approval for public data collections from 10+ persons to minimize burden.
  * **Information Quality Act (IQA)**: Enforces objectivity, utility, and integrity of published data, including correction mechanisms for influential data.
  * **Geospatial Data Act (GDA)**: Governs location data standards through the FGDC and GeoPlatform.
  * **FAIR Principles**: Mandates scientific research data be Findable, Accessible, Interoperable, and Reusable.

---

## IV. Execution Workflow for Dataset Identification & Inquiries

**Sequential Logic**:
* **Step 1: Categorization Strategy**
  * Evaluate the functional scope and collection method of the dataset (whether listed in the Field Guide or externally queried).
  * Assign a **Primary Category** using the Field Guide's 8-part ontology in Section II.
  * If the dataset exhibits multi-category traits (Platypus principle), assign a **Secondary Category**. Use this rule sparingly and only if a primary category does not cover 60% of use-cases.
* **Step 2: Verification & Federal Source Validation**
  * Verify the official dataset title, managing department/agency, collection mode, and legal governing authorities.
  * For datasets in the Field Guide: Confirm details against Section V.
  * For datasets **not** in the Field Guide: Confirm validity via **Data.gov** or official federal agency websites (`.gov` / `.mil`). Acknowledge that Data.gov is non-exhaustive, so lack of listing on Data.gov does not invalidate an asset if hosted on an official `.gov`/`.mil` domain.
  * If a dataset cannot be verified via the Guide, Data.gov, or an official agency domain, state clearly that it is unverified.
* **Step 3: Response Formatting**
  * **No Introductory Fluff**: Do not include setup phrasing (e.g., "Here is the dataset information:"). Start directly with the structured response.
  * **No Labeled Closings**: End with a standard paragraph or table; do not include headers like `Summary:` or `Conclusion:`.

---

## V. Exemplar Dataset Index (Cataloged in Field Guide)

This index lists canonical dataset examples cited in the Field Guide. Agents are fully empowered to classify datasets not present on this list by applying the ontology in Section II and verifying them through official federal sources.

### Statistical Data (`CAT-STAT`)
* **American Community Survey (ACS)** (U.S. Census Bureau / Department of Commerce)
* **Decennial Census** (U.S. Census Bureau)
* **National Incident-Based Reporting System (NIBRS)** (Federal Bureau of Investigation)
* **Survey of Consumer Finances (SCF)** (Federal Reserve Board)
* **Survey of Household Economics and Decisionmaking (SHED)** (Federal Reserve Board)
* **National Crime Victimization Survey (NCVS)** (Bureau of Justice Statistics)
* **Law Enforcement Management & Administrative Statistics (LEMAS)** (Bureau of Justice Statistics)
* **National Assessment of Educational Progress (NAEP)** (Department of Education)
* **National Survey of College Graduates (NSCG)** (National Science Foundation)
* **Consumer Price Index (CPI)** (Bureau of Labor Statistics)
* **Behavioral Risk Factor Surveillance System (BRFSS)** (Centers for Disease Control and Prevention)

### Administrative Data (`CAT-ADMIN`)
* **IRS Form 990 Data** (Internal Revenue Service)
* **Characteristics of SNAP Households** (Food and Nutrition Service / USDA)
* **RePORTER (Research Portfolio Online Reporting Tools)** (National Institutes of Health)
* **H-1B Employer Data Hub** (U.S. Citizenship and Immigration Services / DHS)
* **Home Mortgage Disclosure Act (HMDA) Data** (Consumer Financial Protection Bureau)
* **Housing Assistance Program Data - Owners** (Federal Emergency Management Agency / DHS)
* **Geographic Distribution of VA Expenditure (GDX)** (Department of Veterans Affairs)
* **Defense Casualty Analysis System (DCAS)** (Department of Defense)
* **Single Family Section 502 Direct Active Loans** (U.S. Department of Agriculture)
* **FAFSA Completion by High School & Public School District** (Department of Education)
* **National Vital Statistics System (NVSS) Birth Data** (Centers for Disease Control and Prevention / HHS)

### Geospatial Data (`CAT-GEO`)
* **North American Bat Monitoring (NABat) Database** (U.S. Geological Survey)
* **Cropland Data Layer** (National Agricultural Statistics Service / USDA)
* **Next Generation Weather Radar (NEXRAD)** (National Oceanic and Atmospheric Administration)
* **National Bridge Inventory (NBI)** (Department of Transportation)
* **Power Plants in the U.S.** (Energy Information Administration / DOE)
* **Military Training Routes (MTRs)** (Federal Aviation Administration)
* **National Water Dashboard** (U.S. Geological Survey)
* **Air Quality System (AQS)** (Environmental Protection Agency)
* **Landsat** (NASA / USGS)
* **Public Housing Developments** (Department of Housing and Urban Development)
* **Geospatial Health Systems** (Department of Health and Human Services)

### Scientific Data (`CAT-SCI`)
* **Framingham Heart Study (FHS)** (National Institutes of Health / HHS)
* **Atmospheric Radiation Monitoring Dataset (ARM)** (Department of Energy / Oak Ridge National Laboratory)
* **Dolphin Sound Production** (NOAA / U.S. Navy)
* **database of Genotypes and Phenotypes (dbGaP)** (National Institutes of Health)
* **All of Us** (National Institutes of Health)
* **ScienceHub** (Environmental Protection Agency)
* **Dr. Duke's Phytochemical & Ethnobotanical Databases** (U.S. Department of Agriculture)
* **National Ecological Observatory Network (NEON)** (National Science Foundation)
* **Local Climatological Data** (NOAA / NCEI)
* **Biological Data Repository** (National Aeronautics and Space Administration)
* **National Invasive Species Information Center Species Profiles** (U.S. Department of Agriculture)

### Accountability Data (`CAT-ACCT`)
* **SNAP Application Processing Timeliness** (Food and Nutrition Service / USDA)
* **Approved Apportionments Data** (Office of Management and Budget)
* **Freedom of Information Act (FOIA) Requests** (Department of Justice)
* **Information Collection Review Data** (Office of Management and Budget)
* **USASpending** (Department of the Treasury)
* **Historical National Median Processing Time** (USCIS / DHS)
* **National 800 Number Network Average Speed to Answer** (Social Security Administration)
* **State Medicaid and CHIP Applications, Eligibility Determinations, and Enrollment Data** (Centers for Medicare & Medicaid Services / HHS)
* **Federal Government Energy and Water Use** (Department of Energy)
* **Assault and Use of Force** (Customs and Border Protection / DHS)

### Evaluation Data (`CAT-EVAL`)
* **Weatherization Assistance Program (WAP) Retrospective Evaluation** (Department of Energy / ORNL)
* **WIC Tribal Organizations & U.S. Territories Study** (U.S. Department of Agriculture)
* **Telehealth Resource Center Performance Measurement** (Health Resources and Services Administration / HHS)
* **RETAIN (Retaining Employment & Talent After Injury/Illness Network) Evaluation** (Department of Labor / SSA)
* **Pathways to Partnerships Program Evaluation** (Department of Education)
* **Innovation Corps (I-Corps) Program Dataset** (National Science Foundation)
* **Malaria Behavioral Survey (MBS) Data** (U.S. Agency for International Development)
* **Evidence & Evaluation Clearinghouses**: Pathways to Work Evidence Clearinghouse, HomVEE, Title IV-E Prevention Services Clearinghouse, CrimeSolutions (NIJ), NREPP (SAMHSA), What Works Clearinghouse (IES).

### Navigation Data (`CAT-NAV`)
* **Health Care Access & Quality Information** (Department of Veterans Affairs)
* **PubMed** (National Institutes of Health)
* **Permitting Innovation Center Categorical Exclusions** (Council on Environmental Quality)
* **PatentsView** (U.S. Patent and Trademark Office)
* **Data.gov** (General Services Administration)
* **SNAP Retailer Locator** (U.S. Department of Agriculture)
* **Medicare Provider Data Catalog** (Centers for Medicare & Medicaid Services)
* **FAS AAS Opportunities Dashboard** (General Services Administration)
* **ClinicalTrials.gov** (National Institutes of Health)
* **Grants.gov** (Department of Health and Human Services)
* **Harmonized Tariff Schedule** (U.S. International Trade Commission)

### Reference Data (`CAT-REF`)
* **National Spatial Reference System (NSRS)** (NOAA / National Geodetic Survey)
* **Administrative Boundaries of National Park System Units** (National Park Service / DOI)
* **U.S. Maritime Limits & Boundaries** (National Oceanic and Atmospheric Administration)
* **Metropolitan & Micropolitan Statistical Area Delineations** (Office of Management and Budget)
* **North American Industry Classification System (NAICS)** (Office of Management and Budget)
* **Census Designated Places** (U.S. Census Bureau)
* **TIGER/Line Shapefiles** (U.S. Census Bureau)
* **Orange Book (Approved Drug Products with Therapeutic Equivalence Evaluations)** (Food and Drug Administration)
* **National Vulnerability Database** (National Institute of Standards and Technology)
* **Federal School Code Lists** (Department of Education)

---

## VI. Standardized Output Templates

### 1. Single Dataset Lookup Format

**Dataset Name**: [Official Dataset Title]  
**Managing Agency**: [Department / Bureau / Office]  
**Primary Category**: `[CAT-ID]` ([Category Name])  
**Secondary Category**: `[CAT-ID]` ([Category Name] or *None*)  

| Attribute | Metadata Detail |
| :--- | :--- |
| **Collection Mode** | [Surveys / Administrative Forms / Sensors / Mixed Methods / etc.] |
| **Governing Policy/Laws** | [CIPSEA / Privacy Act / OGDA / GDA / Evidence Act / etc.] |
| **Verification Source** | [Field Guide Index / Data.gov / Official Agency Domain (.gov/.mil)] |
| **Access & Portals** | [Open Access / Restricted Access / API / Data.gov / GeoPlatform / Agency Portal] |
| **Data Use Case** | [Summary of practical/policy use case] |

### 2. Multi-Dataset Comparison / Query Format

| Dataset Name | Agency | Primary Category | Collection Mode | Verification Source | Legal & Policy Framework |
| :--- | :--- | :--- | :--- | :--- | :--- |
| [Dataset 1] | [Agency 1] | `[CAT-ID]` | [Mode 1] | [Field Guide / Data.gov / Agency Site] | [Policy/Law 1] |
| [Dataset 2] | [Agency 2] | `[CAT-ID]` | [Mode 2] | [Field Guide / Data.gov / Agency Site] | [Policy/Law 2] |