Evaluation against the Standard
Have your model of a person judged the way ours is: blind, by the people who know that person, with the same three numbers and the same floor. The result is published about the system, never about a person.
- You receive
- A campaign report: the three bands per archive and overall, the publishable flag, the protocol as anchored before the campaign.
- Delivered as
- A console campaign with a signed PDF; trials run on our infrastructure, your model called through your endpoint.
- Requires
- Travelers who said yes to the "judge other models" purpose and are paid for their time; at least five archives; thirty judged items each.
| Archive | Trials | Recog. | Abst. | Conf. wrong | Status |
|---|---|---|---|---|---|
| A-01 | 32 | 77% | 18% | 5% | publishable |
| A-02 | 30 | 65% | 30% | 5% | publishable |
| A-03 | 31 | 71% | 19% | 10% | at ceiling |
| A-04 | 24 | — | — | — | under floor |
| A-05 | 33 | 74% | 22% | 4% | publishable |