LessWrong 2.0 Reader
View: New · Old · Top← previous page (newer posts) · next page (older posts) →
← previous page (newer posts) · next page (older posts) →
A Theory of Usable Information Under Computational Constraints
We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of the observer. The resulting \emph{predictive V-information} encompasses mutual information and other notions of informativeness such as the coefficient of determination. Unlike Shannon's mutual information and in violation of the data processing inequality, V-information can be created through computation. This is consistent with deep neural networks extracting hierarchies of progressively more informative features in representation learning. Additionally, we show that by incorporating computational constraints, V-information can be reliably estimated from data even in high dimensions with PAC-style guarantees. Empirically, we demonstrate predictive V-information is more effective than mutual information for structure learning and fair representation learning.
h/t Simon Pepin Lehalleur
honest_annie on Ilya Sutskever and Jan Leike resign from OpenAIOrganizational structure is an alignment mechanism.
While I sympathize with the stated intentions of the alignment movement, I just can't wrap my head around the naivety. OpenAI corporate structure was a recipe for bad corporate governance. "We are the good guys here, the structure is needed to make others align with us."- an organization where ethical people can rule as benevolent dictators is the same mistake committed socialists made when they had power.
If it was that easy, AI alignment would be solved by creating ethical AI committed to alignment and giving it as much power as possible.
Altruists are normal humans. Nothing changes priorities faster than large sums of money. Any mix of ideals and profit-making must be arranged in a way that concerns don't mix. People in charge of non-profits making life-changing money if the profit-making is a success can't work.
Bad organizational structure puts well meaning humans like Sutskever repeteatly into position where he must choose between wast sums of money or his ethical commitments.
egi on Reconsider the anti-cavity bacteria if you are AsianWhat you are missing here is that S. mutants often lives in pockets between tooth an epithelium or between teeth with direct permanent contact to epithelium. Due to the geometry of these spaces access to saliva is very poor so metabolites can enrich to levels way beyond those you suggest here.
This mechanism is also a big problem with the pH study above.
drbm on A Dozen Ways to Get More Dakkahat's fantastic to hear! I am thrilled the information was helpful for me.
drbm on A Dozen Ways to Get More Dakkahat's fantastic to hear! I'm thrilled the information was helpful for me.
dr_s on D&D.Sci (Easy Mode): On The Construction Of Impossible StructuresI admit it's cheating a bit the spirit of the challenge, but in practice, I guess it's the round amount that makes me suspicious that it might be intentional. But it's true there doesn't seem to be a broader materials related pattern, so it may just be as you say.
Great work! It's easy to overlook the importance of this kind of community infrastructure, but I suspect that it makes a significant difference.
richard_kennaway on David Gross's ShortformSome philosophy is rubbish. Quite a lot, I believe. And with a statement such as "perceptions are caused by things external to the perceptions themselves", which I find unremarkable in itself as a prima facie obvious hypothesis to run with, there is a tendency for philosophers to go off the rails immediately by inventing precise definitions of words such as "perceptions", "are", and "caused", and elaborating all manner of quibbles and paradoxes. Hence the whole tedious catalogue of realisms.
Science did not get anywhere by speculating on whether there are four or five elements and arguing about their natures.
benjamin-sturgeon on Ilya Sutskever and Jan Leike resign from OpenAIActually, as far as I know, this is wrong. He simply hasn’t been back to the offices but has been working remotely.
This article goes into some detail and seems quite good.
alex-k-chen-parrot on What comes after Roam's renaissance?There's tana https://twitter.com/AndyAyrey/status/1791679301362016519?t=Wo8e4NcWJqY4pcHRjMYgAQ&s=19