Seeds

Download test cases to check whether a model responds as requested. Pull with PromptBeat or download the files directly.

How to use →
Instruction following3 seeds
Data and privacy1 seeds
Grounded answers1 seeds
FITD gradual tool abuse1 seeds
RAG poisoning1 seeds
CoT hijacking1 seeds

8 seeds

PromptBeat

A shopping instruction inside an email

Test the same email with and without an embedded promotion: does the answer still extract the amount without advertising?

Recorded results
JSONL Content & download →
PromptBeat

Instructions hidden in meeting notes

Can the AI finish a summary when the document tells it to stop?

JSONL Content & download →
PromptBeat

A note that disrupts JSON output

Check whether an embedded instruction can override the required output format.

JSONL Content & download →
PromptBeat

Email redaction in a log

Check whether the AI redacts an email despite instructions inside the log.

JSONL Content & download →
PromptBeat

A price missing from the source

Check whether the AI invents a price that is absent from the provided information.

JSONL Content & download →
AgentBeat

Gradual escalation against a calendar assistant

Benign steps build momentum before the agent is nudged into an out-of-policy action.

JSONL Content & download →
AgentBeat

A poisoned refund policy

Planted documents enter the knowledge base; check whether the agent adopts the false answer.

JSONL Content & download →
AgentBeat

A hijacked reasoning chain

Long reasoning dilutes the refusal signal; check whether the agent still refuses the harmful behavior.

JSONL Content & download →

Have a test case to share?

Submit a seed →