Introducing Squiggle AI
post by ozziegooen · 2025-01-03T17:53:42.915Z · LW · GW · 11 commentsContents
11 comments
11 comments
Comments sorted by top scores.
comment by ozziegooen · 2025-01-03T18:05:59.366Z · LW(p) · GW(p)
If you'd prefer, feel free to leave questions for Squiggle AI here, and I'll run the app on them and respond with the results.
Replies from: silentbob↑ comment by silentbob · 2025-01-03T21:33:13.619Z · LW(p) · GW(p)
In the EU there's some recent regulation about bottle caps being attached to bottles, to prevent littering. (this-is-fine.jpg)
Can you let the app come up with a good way to estimate the cost benefit ratio of this piece of regulation? E.g. (environmental?) benefit vs (economic? QALY?) cost/drawbacks, or something like that. I think coming up with good metrics to quantify here is almost as interesting as the estimate itself.
Replies from: silentbob↑ comment by silentbob · 2025-01-03T21:50:53.038Z · LW(p) · GW(p)
Or as a possible more concrete prompt if preferred: "Create a cost benefit analysis for EU directive 2019/904, which demands that bottle caps of all plastic bottles are to remain attached to the bottles, with the intention of reducing littering and protecting sea life.
Output:
-
key costs and benefits table
-
economic cost for the beverage industry to make the transition
-
expected change in littering, total over first 5 years
-
QALYs lost or gained for consumers throughout the first 5 years"
↑ comment by ozziegooen · 2025-01-04T18:32:43.402Z · LW(p) · GW(p)
Okay, I ran this a few times.
I tried two variations. In the first, I just copied and pasted the text you provided. In the second, i first asked Perplexity to find relevant information about your prompt - then I pasted this information into Squiggle AI, with the prompt.
Here are the outputs without the Perplexity data:
https://squigglehub.org/models/ai-generated-examples/eu-bottle-directive-172
https://squigglehub.org/models/ai-generated-examples/eu-bottle-directive-166
And with that data:
https://squigglehub.org/models/ai-generated-examples/eu-bottle-directive-with-data-113
https://squigglehub.org/models/ai-generated-examples/eu-bottle-directive-with-data-120
Of the ones with data, they agree that the costs are roughly €5.1B, the QALYs gained are around 30 to 3000, leading to a net loss of around €−5.1B. (They both estimate figures of around €50k to €150k of willingness to pay for a QALY, in which case 30 to 3000 QALYs is very little compared to the cost).
My personal hunch is that this is reasonable as a first pass. That said, I think the estimate of the QALYs gained seems the most suspect to me. These models estimated this directly - they didn't make sub-models of this, and their estimates seem wildly overconfident if this is meant to include sea life.
I think it could make sense to make further models delving more into this specific parameter.
↑ comment by Matt Goldenberg (mr-hire) · 2025-01-04T21:59:54.530Z · LW(p) · GW(p)
i first asked Perplexity to find relevant information about your prompt - then I pasted this information into Squiggle AI, with the prompt.
It'd be cool if you could add your perplexity api key and have it do this for you. a lot of the things i thought of would require a bit of background research for accuracy
Replies from: ozziegooen↑ comment by ozziegooen · 2025-01-05T00:35:07.849Z · LW(p) · GW(p)
Yep, this is definitely one of the top things we're considering for the future. (Not sure about Perplexity specifically, but some related API system).
I think there are a bunch of interesting additional steps to add, it's just a bit of a question of developer time. If there's demand for improvements, I'd be excited to make them.
comment by Nathan Helm-Burger (nathan-helm-burger) · 2025-01-04T06:32:16.022Z · LW(p) · GW(p)
I'm someone who tried squiggle out back when you had recently announced it. I liked it, and could see uses for it, but forgot about it. Since then I've continued to do my thinking/modeling of this style in jupyter notebooks.
I like the AI squiggle better, although it is frustrating in some ways (not making quite what I had in mind). I tried using it to recreate some plots Claude had made for me recently in a conversation about risk modeling. I eventually got most of what I wanted, but more slowly and with more frustration than from just having Claude directly make interactive plots for me (where I can adjust the values with sliders to aee the results on the predictions).
So, I will continue trying to keep squiggle in mind for future tasks and try it again... But I expect Claude is going to get better at making charts directly on its own within a few months... So that's a tough race to be in.
Replies from: ozziegooen↑ comment by ozziegooen · 2025-01-04T18:10:00.608Z · LW(p) · GW(p)
Thanks for the info!
Yea, I think it's a challenge to beat Claude/ChatGPT at many things. Lots of startups are trying now, and they are having wildly varying levels of success.
I think that Squiggle AI is really meant for some fairly specific use cases. Basically, if you want to output a kind of cost-effectiveness model that works well in Squiggle notebooks, and you're focused on estimating things without too much functional complexity, it can be a good fit.
Custom charts can get messy. The Squiggle Components library comes with a few charts that we've spent time optimizing, but these are pretty specific. If you want fancy custom diagrams, you probably want to use JS directly, in which case environments like Claude's make more sense.
↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2025-01-04T18:14:51.393Z · LW(p) · GW(p)
Hmm. I wonder if it would make sense with Squiggle AI to give Claude some space for "freeform" diagrams coded in JS, based on the variables from the code... Might be worth experimenting with. Maybe too soon, but it'd probably work after the next upgrade (e.g. Claude 4)
Replies from: ozziegooen↑ comment by ozziegooen · 2025-01-04T18:19:18.509Z · LW(p) · GW(p)
Yea, I think there's generally a lot of room for experimentation around here.
I very much hope that in the future, AI cools could be much more compositional, so you won't need to work only in one ecosystem to get a lot of the benefits.
In that world, it's also quite possible that Claude could call Squiggle AI for modeling, when is needed.
A different option I see is that we have slow tools like Squiggle AI that make large models that are expected to be useful for people later on. The results of these models, when interesting, will be cached and made publicly available on the web, for tools like Claude.
In general I think we want a world where the user doesn't have to think about or know which tools are best in which situations. Instead that all happens under the hood.
comment by rai (nonveumann) · 2025-01-04T04:39:55.603Z · LW(p) · GW(p)
This is cool. I asked it a question in a field I'm not knowledgable about at all but my roommate is. I think having the structure there to make it maximally easy to elicit his expertise is super valuable.