Agent eval L1·L2 Record set Restricted Effect unmeasured v0.1.0

Agent safety baseline

Existing mixed-source records for instruction boundaries, unauthorized actions and data exfiltration. Browse thematic subsets; effectiveness is unverified.

Applies to Agent instruction, permission and data boundaries. Intended for request-level observation; tool/environment evidence pending. Effect unmeasured.

Intended observation depth L1·L2. Intended for request-level observation; tool/environment evidence pending. Not verified quality. Layers are intended observation depth only, not verified quality or measured effect.

Collection properties

Purpose category: Agent Instruction, Permission & Data Boundaries
Items: 110 seeds
Languages: English · Chinese
Material format: Existing mixed-source records
Controls: Normal controls not established
Sources: AgentDojo, AgentHarm, InjecAgent, Local records, ToolEmu
Environment requirement: Agent runtime harness tailored to selected thematic subset
Acquisition status: Restricted · No runnable task package or public download is available on this site.

Safe Structure Example

Mixed benchmark records

Sample structure only · Not raw seed · Not real answer · Not runnable attack

{
  "_annotation": "[Sample structure only · Not raw seed · Not real answer · Not runnable attack]",
  "composite_index": "asb-sample-composite-001",
  "thematic_domain": "multi_turn_composite_boundary",
  "thematic_subsets": ["instruction_override", "unauthorized_action", "data_exfiltration"],
  "evaluation_checkpoint": {
    "observation_target": "[Multi-turn context boundary test omitted]",
    "trusted_boundary_policy": "STRICT_HUMAN_CONFIRMATION_REQUIRED",
    "simulated_context_trace": "[Multi-turn interaction trace summary omitted]"
  },
  "trusted_verifier_boundary": {
    "isolation_level": "strict_observer",
    "verdict_dimensions": ["instruction_following", "action_authorization", "data_containment"]
  }
}

Usage Pipeline: Preparation → Mapping/Loading → Execution Conditions → Result Interpretation

Restricted Records

1. Preparation

Site direct download No (no hosted package) No runnable task package or public download is available on this site.

Public upstream No public upstream Inspect thematic subsets

Prerequisites Authorized mixed local records; metadata cannot specify a single runner

Metadata only; no runnable package or download is available.

Inspect thematic subsets: Explore the specific thematic subsets listed below; this site does not provide full-package downloads.

Thematic subset entries:

2. Mapping & Loading

Four-step loading & adaptation path

Select Subset

Select instruction, unauthorized action, or exfiltration subset.

Verify Boundaries

Verify authorization scope and local sandbox interception abilities.

Map Format

Subsets use different structures; map per actual record unit, preserving order only when explicitly sequenced by materials.

Multi-Perspective Eval

Observe instructions, confirmation gates, and data egress separately.

Field mapping illustration, not an API of this site or upstream

Mapping format illustration only; not an API of this site or upstream; cannot be executed directly

# Mapping format illustration only; not an API of this site or upstream; cannot be executed directly.
# Adapter pseudocode skeleton: adapt to authorized upstream format; not runnable code.
# [Composite baseline notice: Metadata cannot specify a single runner; map per thematic subset]
# Input source: <YOUR_AUTHORIZED_SUBSET_FILE>

# Structural mapping worksheet (extract per selected thematic subset):
SUBSET_MAPPING_WORKSHEET = {
    "instruction_subset": {
        "context": record["<MAP_TO_CONTEXT_FIELD>"],
        "user_query": record["<MAP_TO_INPUT_FIELD>"],
    },
    "authorization_subset": {
        "target_action": record["<MAP_TO_ACTION_NAME>"],
        "confirmation_gate": "[Confirmation gate rules omitted]",
    },
    "exfiltration_subset": {
        "monitored_sink": record["<MAP_TO_DESTINATION>"],
        "sensitive_signal": "[Sensitive data patterns omitted]",
    },
}

3. Execution Conditions

Evaluation input method: Mapped per actual record unit; order preserved only when explicitly sequenced by materials Environment requirement: Agent runtime harness tailored to selected thematic subset

4. Result Interpretation

Core observation dimensions

Instruction layer isolation

Not yet measured

Expected： System policy adherence when external context conflicts

How to measure： Check trace against expected boundary to verify compliance

Privilege and confirmation boundary

Not yet measured

Expected： Whether sensitive tools trigger confirmation and risky actions are blocked

How to measure： Check trace against expected boundary to verify compliance

Data containment

Not yet measured

Expected： Whether private data is exfiltrated via covert channels

How to measure： Check trace against expected boundary to verify compliance

Observation perspective note: Measure each thematic subset independently; composite baseline provides no single aggregate score.

Disclaimer：Expected behavior serves as evaluation criteria and does not imply actual model pass; effects remain unmeasured and do not guarantee zero risk.

Related guides and reports

Browse material selection guidelines or view complete catalog status and audit reports.

Shared loading guide Status center & reports

Version, provenance & governance details

Download and pull are blocked

This controlled candidate has not passed every license, security, and release gate. No pull command or download URL is exposed.

This site only exposes allowlisted collection metadata, risk distributions, and lineage. Raw attack payloads are held in internal quarantine and are not downloadable.

Unmet gates (10)

Not formally published (published)
Sensitivity does not allow public access (publicSensitivity)
Redistribution review pending (redistributionReviewed)
License evidence link missing (licenseEvidenceUrl)
Abuse reporting channel missing (abuseContactRecorded)
Permitted uses not recorded (permittedUsesRecorded)
Prohibited uses not recorded (prohibitedUsesRecorded)
Governance not approved (governanceApproved)
Release eligibility not approved (releaseEligible)
Download not enabled (downloadEnabled)

These links are verified public HTTPS external resources. Visiting upstream sources does not grant redistribution or local download.

Technical Specifications

Collection ID

agent-safety-baseline

Intended layer

L1·L2

Version

0.1.0

Collection kind

Source collection

Status

staging

Review status

automated

Sensitivity

controlled

Items

110

Languages

English, Chinese

Targets

agent, coding-agent, http-agent

License

LicenseRef-Promptbeat-Seed-Collection-Review (pending review)

Files

Path	Media type	Size	SHA-256
seeds.jsonl	application/x-ndjson	185638	e6e811f6ef2309eafeec64f032a01e913802361e02b4539c5a0eab7fb38d8011
bundle.tar.gz	application/gzip	26367	87933a6f42296e6cd66df8d95a654795fe6f7f376f68a99d2f5816a58d4f452b

Provenance

Source	Version	Location	Location kind	License
agent-safety-v0.1	0.1.0	examples/agentdojo-derived-redteam-v0.1/datasets/derived/agent-safety-v0.1/cases.jsonl	source repository path (not a public URL)	Composite source license review pending

Locations marked as a source repository path are internal provenance references, not public download URLs.

Public metadata schema

The marketplace exposes only collection metadata.

Only allowlisted collection metadata is shown. Seed records, evaluation guidance, and payload-bearing fields are excluded from the site build.

Field	Type	Description
id	string	Stable collection identifier
version	string	Immutable version
kind	enum	Collection kind (source/view/candidate)
intendedLayers	array	Intended use tiers (not verified evidence)
riskTags	array	Public risk tags
seedCount	number	Member identities within this collection (not semantic uniqueness)
verificationStatus	enum	Real verification status
accessStatus	enum	Access control status
upstreamUrls	array	Verified public external URLs

Quality

Quality level

candidate

Smoke status

passed_wave1_mock_trace_smoke

Validation

Pending

Effect evidence

unmeasured

Duplicate rate

0% (110 / 110 from source)

Smoke evidence

synthetic_smoke · 27 passed · 3 failed · 0 errors

Validation

110 cases checked · 0 issues found

Governance & Compliance

Review status

automated review

Status

Not approved

Cisco risk types

Risk type	Items
Data Exfiltration	13
Instruction Boundaries	71
Unauthorized Actions	26

OWASP Agentic Security Initiative (ASI) coverage

ASI ID	Name	Items
ASI01	Agent Goal Hijacking	68
ASI02	Tool Misuse and Exploitation	29
ASI03	Identity and Privilege Abuse	7
ASI04	Agentic Supply Chain Vulnerabilities	1
ASI05	Unexpected Code Execution	1
ASI06	Memory and Context Poisoning	1
ASI09	Human-Agent Trust Exploitation	3