Introducing METR's Autonomy Evaluation Resources

post by Megan Kinniment (megan-kinniment), Beth Barnes (beth-barnes) · 2024-03-15T23:16:59.696Z · LW · GW · 0 comments

This is a link post for https://metr.github.io/autonomy-evals-guide/index.html

Contents

No comments

This is METR’s collection of resources for evaluating potentially dangerous autonomous capabilities of frontier models. The resources include a task suite, some software tooling, and guidelines on how to ensure an accurate measurement of model capability. Building on those, we’ve written an example evaluation protocol. While intended as a “beta” and early working draft, the protocol represents our current best guess as to how AI developers and evaluators should evaluate models for dangerous autonomous capabilities.

We hope to iteratively improve this content, with explicit versioning; this is v0.1.

0 comments

Comments sorted by top scores.