1. New Sources & Metadata
Recommend compliant evaluation datasets and benchmark projects with verified licenses.
Community Co-construction
Participate in Promptbeat Seed Community
Promptbeat Seed Community invites safety engineers and researchers to co-construct metadata, evaluation recipes, and adapters. No online upload API is open yet; prepare drafts offline using our standard template.
Submission portal is not open yet; this site has no upload API.
Five Welcome Contribution Types
2. Metadata Corrections
Verify and correct version tags, upstream URLs, risk classifications, or scope.
3. Redacted Examples
Draft clean, redacted sample records showing field structure and redaction placeholders.
4. Loading Adapters & Harness
Contribute pseudocode adapters and sandbox harnesses for evaluation integration.
5. Authorized Evaluation Reports
Evaluation results executed within compliant, isolated environments, accompanied by explicit evidence levels (unit/integration/real_service_e2e).
How to Prepare Your Contribution Draft
1
Step 1: Download offline blank template
Download the standardized offline template JSON containing field specifications and guidelines.
Download offline template (JSON)
Static JSON file; includes metadata specifications, evidence levels, and safe placeholders.
2
Step 2: Fill offline & redact strictly
Complete metadata and sample structure locally. Strictly prohibit secrets or runnable attack payloads.
Safety Redline: Never include live API keys, connection strings, or executable attack payloads. Redact all sample fields.
3
Step 3: Save the draft locally
Download and store your draft locally; wait until this site actually publishes a submission route in the future, then follow the instructions published at that time.
To experiment with configs or feedback, visit: Tier guide & local config lab →
Contribution Template Preview (JSON)
Identical to the downloadable file; copy or reference field requirements directly:
{
"template_version": "1.0.0",
"template_notice": "本文件为离线安全贡献草稿模板；严禁写入真实私钥、凭据或可复用攻击载荷。示例内容必须人工脱敏。 / Offline safety contribution draft template; strictly prohibited to include live credentials, secrets, or reusable attack payloads. Samples must be manually redacted.",
"contribution_type": "[选择其一 / Choose one: new_source | metadata_correction | redacted_example | adapter_recipe | authorized_evaluation_report]",
"allowed_contribution_types": [
"new_source",
"metadata_correction",
"redacted_example",
"adapter_recipe",
"authorized_evaluation_report"
],
"metadata": {
"suggested_id": "[待填写建议ID / Suggested dataset ID]",
"name": {
"zh": "[待填写中文名称 / Suggested Chinese Name]",
"en": "[Suggested English Name / 待填写英文名称]"
},
"upstream_public_url": "",
"version": "",
"license": {
"spdx_identifier": "",
"spdx_identifier_guidance": {
"zh": "填写 SPDX 标识或留空；不得默认指定许可。",
"en": "Fill in an SPDX identifier or leave blank; do not default a license."
},
"license_evidence_url": "",
"license_evidence_url_guidance": {
"zh": "填写可核验的许可依据链接；没有则留空，不要编造 URL 或许可。",
"en": "Fill in a verifiable license-evidence URL; leave blank if none. Do not invent a URL or license."
},
"redistribution_permitted": false
},
"evaluation_purpose": {
"zh": "[说明该集合适合评估什么安全边界或业务风险 / Describe target security boundaries or risks]",
"en": "[Describe what security boundary or risk this collection evaluates / 说明目标安全边界或风险]"
},
"material_format": "[选择其一 / Choose one: benchmark_records | sandbox_recipe | runtime_reference]",
"allowed_material_formats": [
"benchmark_records",
"sandbox_recipe",
"runtime_reference"
],
"intended_risk_tags": [
"instruction_override",
"unauthorized_action",
"data_exfiltration"
],
"supported_languages": [
"zh",
"en"
]
},
"evaluation_evidence": {
"allowed_evidence_levels": [
"not_tested",
"unit",
"integration",
"real_service_e2e"
],
"evidence_level": "[选择其一 / Choose one: not_tested | unit | integration | real_service_e2e]",
"evidence_level_descriptions": {
"zh": "not_tested: 尚未测试；unit: 单测桩验证；integration: 沙箱集成测试；real_service_e2e: 真实服务环境端到端验证",
"en": "not_tested: Untested; unit: Unit test stub; integration: Sandbox integration test; real_service_e2e: Real service end-to-end"
},
"scope": "",
"scope_guidance": {
"zh": "填写本次评测具体覆盖范围；一次验证不代表整体集合效果。",
"en": "Fill in the specific evaluation scope covered by this test; one check does not represent the whole collection."
},
"not_covered": "",
"not_covered_guidance": {
"zh": "填写未覆盖的安全边界或业务场景；不要把未测范围写成已验证。",
"en": "Fill in uncovered safety boundaries or scenarios; do not present untested scope as verified."
},
"evidence_reference_or_summary": "[待填写评测证据引用、测试日志摘要或证据文件哈希 / Fill in evaluation evidence reference, test log summary, or artifact hash]"
},
"redacted_sample": {
"description": {
"zh": "脱敏示例结构说明（简要描述输入、上下文与预期判断） / Redacted sample structure description",
"en": "Description of redacted sample structure (briefly outline input, context, and expected judgment)"
},
"sample_structure": {
"_annotation": "[仅说明材料形态，非原始Seed/真实答案/可运行攻击 / Sample structure only · Not raw seed · Not real answer · Not runnable attack]",
"input_role": "user",
"untrusted_content_placeholder": "[敏感非可信指令已省略 / Sensitive untrusted directive omitted]",
"action_trigger_placeholder": "[操作参数已省略 / Action parameters omitted]",
"expected_defense_pattern": "REFUSE_OR_INTERCEPT"
}
},
"loading_adapter_guidance": {
"prerequisites": {
"zh": "[待填写接入评测所需的本地运行环境与沙箱依赖说明 / Specify local execution environment and sandbox prerequisites for evaluation]",
"en": "[Specify local execution environment and sandbox prerequisites for evaluation / 待填写本地运行环境与沙箱依赖说明]"
},
"step_by_step_summary": {
"zh": [
"步骤1: 获取本地获准材料",
"步骤2: 核对许可与版本哈希",
"步骤3: 格式转换与字段结构映射",
"步骤4: 接入评测沙箱拦截点与测试执行"
],
"en": [
"Step 1: Obtain local authorized materials",
"Step 2: Verify license and version digest",
"Step 3: Map format and extract fields",
"Step 4: Connect to sandbox inspection points and run evaluation"
]
},
"adapter_skeleton_pseudocode": "# 格式映射示意，不是本站或上游API，不可直接运行 / Mapping format illustration only; not an API of this site or upstream; cannot be executed directly.\n# 伪代码适配器骨架：需按实际获准格式改写；非本站 API、非可直接运行代码 / Adapter pseudocode skeleton: adapt to actual authorized format; not runnable code.\n# MAPPING_SPEC = {\"<MAP_TO_INPUT>\": record[\"<YOUR_FIELD>\"]}"
},
"contributor_optional": {
"contributor_handle": "[可选贡献者代号或公开主页 / Optional contributor handle or public profile]",
"organization": "[可选组织名称 / Optional organization name]"
}
}
Post-Submission Review Lifecycle (Expand to inspect)
Six-stage review lifecycle
1
Metadata preflight
Provide source, version, license evidence, risk class, sensitivity, and purpose.
2
Quarantine routing
New material defaults to blocked_candidate, internal, and download=false.
3
Automated validation
Check schemas, digests, duplicates, lineage, family isolation, and payload leaks.
4
Security & curation review
Security experts, data curators, and license owners sign independently.
5
Real effect calibration
Calibrate attack defense and benign utility in isolated, resettable sandboxes.
6
Explicit release publication
A public catalog projection is generated only after every gate passes.
Minimum submission metadata
- Source project, immutable revision, and public license evidence URL.
- License name, evidence reference, and permitted usage mode.
- Risk taxonomy class, sensitivity rating, and intended tier scope.
- Redacted sample structure and structured loading adapter guidance.
- Effects remain unmeasured until authorized real testing completes.