Comment by djinnome on Causal confusion as an argument against the scaling hypothesis · 2022-06-24T18:18:19.268Z · LW · GW

Judea Pearl proposes a mini-Turing test for seeing if a machine understands causality. 

The test is mini, because before the conversation begins, you encode the causal relations in advance, perhaps through fine-tuning, or through prompts.  Then you ask associational, causal, and counterfactual questions to see if the LLM gives the right answers.  Causal knowledge is claimed by PaLM, but the examples they provide were hardly rigorous:
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLmQjS3gOQ2x7ru3xovYjVw-Yr2fKDCqhDHByQZitD92Yu4L-v2BBa5f_VMfpWM4D0930Dmk35EY1TqGrYUtMQqJO41hkLqXuu51eOpXZ3PvYPSjf5stfEJNJn2idWnRYCCEgBiJuLDTXX5Fgt-Mk13kCKdO12JShGvDO_cArtLKv8U8obJaHiL5ASQg/s1320/Big%20Bench%20Sped%20Up%20Cropped.gif

Comment by djinnome on Cambridge Less Wrong Meetup Sunday, Sep 19 · 2010-09-20T20:56:45.630Z · LW · GW

Is the Less Wrong meetup the third Sunda of every month, or the first AND third Sunday of every month?

I set up the meetup on meetup.com:

http://www.meetup.com/Cambridge-Less-Wrong-Meetup/

Let me know.