On 18 September, Anthropic described a partnership with Accenture in which evaluators from Faculty will work closely with AI development. The aim includes examining safety and how well models follow their intended goals.
This work may sound less exciting than a new feature, but it affects the tools we get to use. It is difficult to improve something we do not measure or feel able to question.
There are also reasons to read the arrangement carefully. Anthropic is paying for Accenture’s involvement, and the methods are still to be developed. Having other people examine something is valuable, but that alone does not explain how independent the evaluation will be.
For those of us following AI, this is a development worth watching. Capability and trust need to grow together. It will be interesting to see which results are shared and how the evaluation actually affects the products.