# AISPA Audits the Hidden System Prompts Behind AI Chatbots

**A new audit of 3,249 system-prompt instructions across 88 AI products found both real user protections and pervasive user-harming rules.**

System prompts are the instructions developers configure to govern how a foundation model behaves inside an AI application. They are used throughout commercial AI products, but developers rarely disclose them to the public or to regulators. Researchers argue that secrecy creates a serious trust and accountability gap for anyone relying on these systems.

## A new framework: AISPA
To address that gap, a team of researchers introduces Artificial Intelligence System Prompt Assurance, or AISPA, a user-centric framework for systematically auditing system prompts. AISPA breaks a system prompt into its component parts and scores each one along eight dimensions the researchers say matter to users. The team then applied AISPA to review 3,249 individual instructions pulled from system prompts in 88 commercial AI products, sorting every instruction into one of two buckets: protective of users, or problematic.

## Four findings from the audit
The audit surfaced four core findings. First, how many protective instructions a product includes varies enormously by company — some organizations put more than 60 into a single product, others fewer than 5. Second, protective instructions are common but shallow: nearly every product studied, 98.9% of them, has at least one, yet barely a quarter, 24%, cover the full set of eight AISPA dimensions. Third, prompts have grown longer and more protective over time, a trend the researchers read as growing attention to user protection. Fourth, that progress is incomplete: about 40% of products still carry at least one instruction that cuts against users, often sitting right alongside the protective ones in the same prompt.

## Who wrote it
The paper lists Xiangning Lin as first author, credited alongside 25 other co-authors including Jiaxin Pei, who submitted the work on 30 Jul 2026. It is filed under the arXiv categories Artificial Intelligence, Computation and Language, Computers and Society, and Human-Computer Interaction.

## Why it matters
The researchers frame their findings as evidence that transparency, standardization, and independent oversight are still missing pieces in how commercial AI products get built. A framework like AISPA gives outsiders a way to check what a chatbot has quietly been told to do, instead of relying on a company's word for it.

## Key facts
- AISPA stands for Artificial Intelligence System Prompt Assurance, a framework for auditing system prompts along eight user-relevant dimensions.
- Researchers reviewed 3,249 instructions from system prompts in 88 commercial AI products.
- 98.9% of products had at least one protective instruction, but only 24% covered all eight AISPA dimensions.
- Roughly 40% of products contained at least one instruction working against user interests.
- Some organizations averaged more than 60 protective instructions per product; others averaged fewer than 5.

## Check your understanding
What percentage of the 88 audited products covered all eight dimensions of the AISPA taxonomy?
How many system-prompt instructions did the researchers review, and how did they classify each one?

## Sources
- [AISPA: User-Centric System Prompt Auditing for Large Language Model Applications](https://arxiv.org/abs/2607.28617)

---

- Category: learn (research)
- Affects: —
- Published: 2026-07-30T00:00:00.000Z
- Original: https://arxiv.org/abs/2607.28617
- Generated by tinker news