Comment by FrancescaG on Are we dropping the ball on Recommendation AIs? · 2024-11-01T11:46:03.104Z · LW · GW

I think incentives. Based on my recent reading of 'The Chaos Machine' by Max Fisher, I think it's closely linked to continually increasing engagement driving profit. Addictiveness unfortunately leads to more engagement, which in turn leads to profit. Emotive content (clickbait style, extreme things) also increase engagement.

Tools and moderation processes might be expensive on their own, but I think it's when they start to challenge the underlying business model of 'More engagement = more profit' that the companies are in a more uncomfortable position.

Comment by FrancescaG on Are we dropping the ball on Recommendation AIs? · 2024-11-01T11:38:56.950Z · LW · GW

I agree that more research on effect of recommendation algorithms on the brain would be useful. 

Also research looking at which cognitive biases and preferences the algorithms are exploiting, and who is most susceptible to these e.g. children, neurodiverse etc. It seems plausible to me that some ai applications e.g. character.ai as you say, will be optimising on some sort of human interaction and exploiting human biases and cognitive patterns will be a big part of this.

Comment by FrancescaG on Toward Safety Cases For AI Scheming · 2024-11-01T11:09:42.228Z · LW · GW

Great addition to thinking on safety cases.

I'm curious about the decision to include the time constraint 'Deployment does not pose unacceptable risk within its 1 year lifetime', as I've been considering how safety case arguments remain relevant post deployment.

Is this intended to convey that the system will be withdrawn/ updated in a year or to help reduce the scope of what is being considered in terms of feasibility of inability and control arguments by looking at a short risk evaluation timeline?

On the topic of the scope of the Harm Inability arguments in the paper:

'A Harm inability argument could be absolute or limited to a given deployment setting'

 Given the vast amounts of deployment settings, the developers of an AI model could provide absolute arguments with accompanying warnings/ limitations based on the evaluation outputs (along the lines of usage guidelines for licensed medicines). Organisations deploying the models in novel settings (also higher risk like medical, finance etc) could supplement these with arguments specific to their deployment setting. Any thoughts on this?