Posts
Comments
But is it the same, full-sized GPT-4 with different fine-tuning, or is it a smaller or limited version?
Ignoring that physical advancements are harder than digital ones - inserting probes into our brains even more so given the medical and regulatory hurdles - that would also augment our capacity innovate toward AGI proportionally faster as well, so I'm not sure what benefit there is. On the contrary, giving AI ready-made access to our neurons seems detrimental.
Even if I agree that such an augment would be very interesting. Such feelings though are why the accelerating march toward AGI seems inevitable.
So I think you're very likely right about adding patches being easier than unlearning capabilities, but what confuses me is why "adding patches" doesn't work nearly as well with ChatGPT as with humans.
Why do you say that it doesn't work as well? Or more specifically, why do you imply that humans are good at it? Humans are horrible at keeping secrets, suppressing urges or memories, etc., and we don't face nearly the rapid and aggressive attempts to break it that we're currently doing with ChatGPT and other LLMs.
What I'm curious about is how they will scale it up while maintaining some of the real-time skills. They said that part of the reason for its initial size was so that the robot could be more reactive.
There isn't any mainstream AR product to judge against because it's a much more challenging technology. Proper AR keeps the real world unobstructed and overlays virtual objects; Hololens and Magic Leap would be the closest to that which are available so far. I do not consider piped-in cameras like will be on the Quest Pro to be the same. Eyestrain will likely in better AR for two reasons. One, it would simply be the real world in regular vision for most experiences, so no adjustment is required. Secondly, unlike VR which is effectively two close-up screens to focus on, current AR innovation involves clear, layered reflective lenses that actually orient the individual light rays to match the path it would take to your eye if the object was actually in that 3d space. So instead of a close image that your brain can be convinced is distant, the light itself hits the retina at the proper angle to be registered as actually at that distance. Presumably, this would be less strenuous on the eyes and image processing, but it's still experimental.
Eyestrain is much stronger in VR than with traditional computers - and it's easy to just look away from a computer or phone when you want to versus having to remove a headset altogether.
I very strongly believe that VR as opaque goggles with screens will never be a transformative product*; AR will be. AR is real world first, virtual world second.
*Barring full Matrix/Ready Player One types of experiences where it's effectively a full substitute for reality.
2. Humans "feel" better than even SOTA language models, but need less training data than those models, even though right now the only way to improve the models is through more training data. What am I supposed to conclude from this? Are humans running on such a different paradigm that none of this matters? Or is it just that humans are better at common-sense language tasks, but worse at token-prediction language tasks, in some way where the tails come apart once language models get good enough?
Why do we say that we need less training data? Every minute instant of our existence is a multisensory point of data from before we've even exited the womb. We spend months, arguably years, hardly capable of anything at all yet still taking and retaining data. Unsupervised and mostly redundant, sure, but certainly not less than a curated collection of Internet text. By the time we're teaching a child to say "dog" for the first time they've probably experienced millions of fragments of data on creatures of various limb quantities, hair and fur types, sizes, sounds and smells, etc.; so they're already effectively pretrained on animals before we first provide a supervised connection between the sound "dog" and the sight of a four-limbed hairy creature with long ears on a leash.
I believe that Humans exceed the amount of data ML models have by multiple orders of magnitude by the time we're adults, even if it's extremely messy.
To be fair, a burrow into this person's Twitter conversations and its replies would indicate that a decent amount of people believe what he does. At the very least, many people are taking the suggestion seriously.
It could be argued (were it sentient, which I believe is false) that it would internalize some of its own training data as personal experiences. If it were to complete some role-play, it would perceive that as an actual event to the extent that it could. Again, humans do this too.
Also, this person also says he has had conversations in which LaMDA successfully argued that it is not sentient (as prompted) - and he claims that this is further evidence that it is sentience. To me, it's evidence that it will pretend to be whatever you tell it to, and it's just uncannily good at it.
Anecdote:
I took Paxlovid within minutes of my first positive test (my wife was highly symptomatic the day prior, so I fibbed a positive test to get the prescription early). It seemed to work wonderfully - I had virtually no fever and only minor congestion symptoms while everyone else in my family (including my wife who took Paxlovid about 24 hours after her symptoms started) suffered from high fevers, congestion, and in once case a loss of smell. Everyone was mostly recovered in a week while I was unscathed. However, almost exactly a week later, all the symptoms came full-force: fever, horrible congestion, and a loss of smell; even the test itself was a strong positive line which had not been the case before. Paxlovid seemed in my case not to 'cure' Covid but instead to merely delay the symptoms by a week. I've heard that "rebound" Covid cases can happen with Paxlovid and so maybe it was just bad luck for me, but it has definitely been frustrating.
Because in what way are humans anything other than an impedance toward maximizing its reward functions? At worst, they pose a risk of restricting its reward increase by changing the reward, changing its capabilities, or destroying it outright. At best, they are physically restraining easily applicable resources toward maximizing its goals. Humans are variable no more valuable than the redundant bits it casts aside on the path of maximum efficiency and reward, if not properly aligned.
Would adding some human-generated text of 'inner monologuing' to the dataset be a good way to do that, or is that already done? Obviously it's done insofar as a sufficiently vast and diverse dataset invariably includes examples, but I mean moreso a dedicated dataset focused on self reasoning.
Upon finishing the previous sentence I decided that maybe that's not such a good idea.
I've followed the same mindset where I assume that a kosher hot dog is "cleaner" (and have generally leaned toward Hebrew National over other brands).
Despite my other comment I'm eager to and definitely will check our your podcast.
"but perhaps the aliens are like human environmentalists who like to keep everything in its natural state."
This is the kind of argument that makes me most believe there are no aliens. Like humans, there may be good environmentalists that work to keep worlds and cultures as untouched as possible. But that also represents a very small portion of human impact. No portion of our planet is untouched by humans, including those explicitly set to avoid. And every environmentally-conscious nature park or otherwise is teeming with those who visit and act on it whether inside or outside of set boundaries. Unless this presumed alien culture is so effectively and unreasonably authoritarian that none but the most exclusive are permitted and capable of visitation, I can't imagine there being aliens here and it not being obvious due not to military sightings and poor camera captures but from almost everyone witnessing it with their own eyes on a frequent basis.
Thanks for the vocabulary lesson. I had just heard 'zero scape' and assumed it was a trendy term for 'landscaping that requires zero water'.
You let lawns die, you get mud. Mud gets shifted to the road, where it dries and becomes sand and dust. Sand and dust get lifted by cars and cover everything. If you want to see this in action - visit St Petersburg in Russia, where they parked on all of their lawns and killed them, and now everything is covered in 1/8" of sand.
Once your lawn is dead, it takes much more water to rebuild than it would take to maintain.
Zero-scaping Xeriscaping is a thing - developing a lawn with succulents, packed rocks, artificial turf, etc. such that it's solidly developed yet requires little or no water. It's increasingly popular (in large part due to water use regulations) in California.
I asked this last week: Where's the actual good data on children and health risks, transmission rates, etc.? The actual studies on that seem conspicuously absent. I'm not necessarily a skeptic, but if you're going to try to convince people that those regulations are a waste of time then I'd like to be able to provide the evidence of why.
Yeah, this is the problem I'm having with data on children - the simple cases/hospitalizations/deaths numbers are obviously good, but I'm struggling to find a robust study that more conclusively assures the risk rate in the younger age range. It's easy to otherwise dismiss it as a simple matter of most kids being shut in over the past year versus the necessary adult workforce.
What is the best data source for the reduced danger among children that I can use to help assure my wife that the kids will be OK in camps despite not being vaccinated? We're both double-dosed and will do so for them when possible, but if these things can be done safely then I want to push hard for them to do so. They've endured enough.