Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agent 2024-07-18T17:02:06.179Z

Colour versus Shape Goal Misgeneralization in Reinforcement Learning: A Case Study 2023-12-08T13:18:39.548Z

Comment by Karolis Jucys (karolis-ramanauskas) on Jesse Hoogland's Shortform · 2025-02-15T06:24:25.912Z · LW · GW

DreamerV3 is not a great example, as they use so many hacks to make the task easier that it barely counts as getting a diamond or Minecraft anymore. Action shaping, macro actions, instant block breaking, fake "bug fixing", all to get a diamond in 0.4% of episodes.

More info here: https://x.com/Karolis_Ram/status/1785750372394348632

Comment by Karolis Jucys (karolis-ramanauskas) on How should TurnTrout handle his DeepMind equity situation? · 2023-10-29T22:23:25.037Z · LW · GW

Would "delta hedging" be useful here? It helps hedge long option exposure by shorting some amount of a stock.
For example, at the money calls generally have a delta of 0.5, so holding 100 at the money calls and shorting 50 shares makes you roughly neutral for small moves in the underlying asset.
Would probably require monthly rebalancing based on how many options you effectively hold and market moves. It also wouldn't work well if AGI happens at GDM and Google stock goes exponential ("volatility smile" problem).

Comment by Karolis Jucys (karolis-ramanauskas) on DeepMind: Model evaluation for extreme risks · 2023-05-26T12:46:38.144Z · LW · GW

non pdf arxiv link: https://arxiv.org/abs/2305.15324

Comment by Karolis Jucys (karolis-ramanauskas) on Polaris, Five-Second Versions, and Thought Lengths · 2022-09-26T21:01:53.662Z · LW · GW

For the four examples of
24-16=12, 53-25=25, 34-16=13, 63-17=16
is this the pattern?