Posts
Comments
Vulnerable world hypothesis (but takeover risk rather than destruction risk). That + first mover advantage could stop things pretty decisively without requiring ASI alignment
As an example, taking over most networked computing devices seems feasible in principle with thousands of +2SD AI programmers/security-researchers. That requires an Alpha-go level breakthrough for RL as applied to LLM programmer-agents.
One especially low risk/complexity option is a stealthy takeover of other AI lab's compute then faking another AI winter. This might get you most of the compute and impact you care about without actively pissing off everyone.
If more confident in jailbreak prevention and software hardening, secrecy is less important.
First mover advantage depends on ability to fix vulnerabilities and harden infrastructure to prevent a second group from taking over. To the extent AI is required for management, jailbreak prevention/mitigation will also be needed.
Slower is better obviously but as to the inevitability of ASI, I think reaching top 99% human capabilities in a handful of domains is enough to stop the current race. Getting there is probably not too dangerous.
Current ATGMs poke a hole in armor with a very fast jet of metal (1-10km/s). Kinetic penetrators do something similar using a tank gun rather than specially shaped explosives.
"Poke hole through armor" is the approach used by almost every weapon. A small hole is the most efficient way to get to the squishy insides. Cutting a slot would take more energy. Blunt impact only works on flimsy squishy things. A solid shell of armor easily stopped thrown rocks in antiquity. Explosive over-pressure is similarly obsolete against armored targets.
TLDR:"poke hole then destroy squishy insides" is the only efficient strategy against armor.
Modern vehicles/military stuff are armored shells protecting air+critical_bits+people
Eliminate the people and the critical bits can be compacted. The same sized vehicle can afford to split critical systems into smaller distributed modules.
Now the enemy has make a lot more holes and doesn't know where to put them to hit anything important.
This massively changes offense/defence balance. I'd guess by a factor of >10. Batteries have absurd power densities so taking out 75% of a vehicle's batteries just reduces endurance. Only way to get a mobility kill is to take out wheels.
There are still design challenges:
- how to avoid ammo cook-off and chain reactions.
- misdirection around wheel motors (improves tradeoffs)
- efficient manufacturing
- comms/radar (antenna has to be mounted externally)
Zerg rush
Quantity has a quality of its own. Military vehicles are created by the thousands, cars by the millions. Probably something similarly sized or a bit smaller, powered by an ICE engine and mass produced would be the best next gen option.
EMP mostly affects power grid because power lines act like big antennas. Small digital devices are built to avoid internal RF like signals leaking out (thanks again FCC) so EMP doesn't leak in very well. DIY crud can be done badly enough to be vulnerable but basically run wires together in bundles out from the middle with no loops and there's no problems.
Only semi-vulnerable point is communications because radios are connected to antennas.
Best option for frying radios isn't EMP, but rather sending high power radio signal at whatever frequency antenna best receives.
RF receiver can be damaged by high power input but circuitry can be added to block/shunt high power signals. Antennas that do both receive and transmit (especially high power transmit) may already be protected by the "switch" that connects rx and tx paths for free. Parts cost would be pretty minimal to retrofit though. Very high frequency or tight integration makes retrofitting impractical. Can't add extra protection to a phased array antenna like starlink dish but it can definitely be built in.
Also front-line units whose radios facing the enemy are being fried are likely soon to be scrap (hopefully along with the thing doing the frying).
RF jamming, communication and other concerns
TLDR: Jamming is hard when comms system is designed to resist it. Civilian stuff isn't but military is and can be quite resistant. Frequency hopping makes jamming ineffective if you don't care about stealth. Phased array antennas are getting cheaper and make things stealthier by increasing directivity.(starlink terminal costs $1300 and has 40dbi gain). Very expensive comms systems on fighter jets using mm-wave comms and phased array antennas can do gigabit+ links in presence of jamming undetected.
civilian stuff is trivial to jam
- EG:sending disconnection messages to disconnect wifi devices requires very little power
- most civvy stuff sends long messages, if you see the start of a message you can "scream" very loudly to disrupt part of it and it gets dropped.
- Civvy stuff like WIFI BT and cellular has strict transmit power limits typically <1W of transmit power.
- TLDR: jamming civvy stuff requires less power than transmitting it. Still, amplifiers and directional antennas can help in the short term.
- military stuff hops from one frequency to another using a keyed unpredictable algorithm.
- Sender and receiver have synchronized clocks and spreading keys so know what frequency to use when. Hop time is short enough jammer can't respond in time.
Fundamentals of Jamming radio signals (doesn't favor jamming)
- Jammer fills big chunk of radio spectrum with some amount of watts/MHz of noise
- EG:Russian R-330ZH puts out 10KW from 100MHz to 2GHz (approx 5KW/GHz or 5W/MHz)
- more than enough to drown out civvy comms like wifi that use <<1W signal spanning 10-100MHz of bandwidth even with short link far away from jammer.
- Comms designed to resist jamming can use 10W+ and reduce bandwidth of transmission as much as needed at cost of less bits/second.
- low bandwidth link (100kb/s) with reasonable power budget is impossible to jam practically until jammer is much much closer to receiver than transmitter.
- GPS and satcom signals easy to jam because of large distance to satellite and power limits.
- Jamming increases required power density to get signal through intelligibly. Transmitter has to increase power or use narrower transmit spectrum. Fundamentally signal to noise ratio decreases and Joules/bit increases.
Communication Stealth
- Jammer + phased array antennas + very powerful computer gives ability to locate transmitters
- Jammer forces transmitters to use more power
- Phased array antennas + supercomputer:
- computer calculates/subtracts reflected jamming signal
- Phased array antenna+computer acts like telescope to find "dimmer" signals in background noise lowering detection threshold
- Fundamental tradeoff for transmitter
directional antennas/phased arrays
- military planes use this to communicate stealthily
- increases power sent/received to/from particular direction
- bigger antenna with more sub-elements increases directionality/gain
- Starlink terminals are big phased array antennas
- this quora answer gives some good numbers on performance
- Starlink terminal gives approx 3000x (35dbi) more power in chosen direction vs omnidirectional antenna
- Nessesary to communicate with satellite 500+km away
- Starlink terminals are pretty cheap
- smaller phased arrays for drone-drone comms should be cheaper.
- drone that is just a big Yagi antenna also possible and ludicrously cheap.
- stealthy/jam immune comms for line of sight data links at km ranges seem quite practical.
- this quora answer gives some good numbers on performance
development pressure for jam resistant comms and associated tech
- little development pressure on civvy side B/C FCC and similar govt. orgs abroad shut down jammers
- military and satcom will drive development more slowly
- FCC limits on transmit power can also help
- Phased array transmit/receive improves signal/noise
- This is partly driving wifi to use more antennas to improve bandwidth/reliability
- hobbyist drone scene could also help (directional antennas for ground to drone comms without requiring more power or gimbals)
Self driving cars have to be (almost)perfectly reliable and never have an at fault accident.
Meanwhile cluster munitions are being banned because submunitions can have 2-30% failure rates leaving unexploded ordinance everywhere.
In some cases avoiding civvy casualties may be a similar barrier since distinguishing civvy from enemy reliably is hard but militaries are pretty tolerant to collateral damage. Significant failure rates are tolerable as long as there's no exploitable weaknesses.
Distributed positioning systems
Time of flight distance determination is in some newer Wifi chips/standards for indoor positioning.
Time of flight across a swarm of drones gives drone-drone distances which is enough to build a very robust distributed positioning system. Absolute positioning can depend on other sensors like cameras or phased array GPS receivers, ground drones or whatever else is convenient.
Overhead is negligible because military would use symmetric cryptography. Message authentication code can be N bits for 2^-n chance of forgery. 48-96 bits is likely sweet spot and barely doubles size for even tiny messages.
Elliptic curve crypto is there if for some reason key distribution is a terrible burden. typical ECC signatures are 64 bytes (512 bits) but 48 bytes is easy and 32 bytes possible with pairing based ECC. If signature size is an issue, use asymmetric crypto to negotiate a symmetric key then use symmetric crypto for further messages with tight timing limits.
Current landmines are very effective because targets are squishy/fragile:
- Antipersonnel:
- take off a foot
- spray shrapnel
- Antitank/vehicle:
- cut track /damage tires
- poke a hole with a shaped charge and spray metal into vehicle insides
Clearing an area for people is hard
-
drones can be much less squishy
- need more explosives to credibly threaten them
-
Eliminating mine threat requires
- clearing a path (no mines buried under transit corridor)
- mine clearing vehicle
- use line charge
- block sensors so off route mines can't target vehicles
- Inflatable barriers that block line of sight/radar
- clearing a path (no mines buried under transit corridor)
This is enough to deal with immobile off route mines. If the minefield has active sensors, those can be spoofed and/or destroyed or blocked at slightly higher expense. Past this, the mines have to start moving to be a threat and then you're dealing with drones vs. drones, not mines.
Ideal mine clearing robots and drones in general should be very resilient:
- No squishy center like people filled vehicles.
- battery powered drone with per wheel motors and multi-part battery pack has no single point of failure.
- Doing meaningful damage to such a drone is hard.
- flimsy exterior can hide interior parts from inspection/targeting.
- Vulnerable systems with fluid like cooling/hydraulics can include isolation valves and redundancy.
- alternatively, no fluids, air for cooling and electric motors/generators/batteries?
- multiple locations/configurations for important components that can be moved (EG:battery/computers)
I think GPT-4 and friends are missing the cognitive machinery and grid representations to make this work. You're also making the task harder by giving them a less accessible interface.
My guess is they have pretty well developed what/where feature detectors for smaller numbers of objects but grids and visuospatial problems are not well handled.
The problem interface is also not accessible:
- There's a lot of extra detail to parse
- Grid is made up of gridlines and colored squares
- colored squares of fallen pieces serve no purpose but to confuse model
A more accessible interface would have a pixel grid with three colors for empty/filled/falling
Rather than jump directly to Tetris with extraneous details, you might want to check for relevant skills first.
- predict the grid end state after a piece falls
- model rotation of a piece
Rotation works fine for small grids.
Predicting drop results:
- Row first representations gives mediocre results
- GPT4 can't reliably isolate the Nth token in a line or understand relationships between nth tokens across lines
- dropped squares are in the right general area
- general area of the drop gets mangled
- rows do always have 10 cells/row
- column first representations worked pretty well.
I'm using a text interface where the grid is represented as 1 token/square. Here's an example:
0 x _ _ _ _ _
1 x x _ _ _ _
2 x x _ _ _ _
3 x x _ _ _ _
4 _ x _ _ o o
5 _ _ _ o o _
6 _ _ _ _ _ _
7 _ _ _ _ _ _
8 x x _ _ _ _
9 x _ _ _ _ _
GPT4 can successfully predict the end state after the S piece falls. Though it works better if it isolates the relevant rows, works with those and then puts everything back together.
Row 4: _ x o o _ _
Row 5: _ o o _ _ _
making things easier
- columns as lines keeps verticals together
- important for executing simple strategies
- gravity acts vertically
- Rows as lines is better for seeing voids blocking lines from being eliminated
- not required for simple strategies
Row based representations with rows output from top to bottom suffer from prediction errors for piece dropping. Common error is predicting dropped piece square in higher row and duplicating such squares. Output that flips state upside down with lower rows first might help in much the same way as it helps to do addition starting with least significant digit.
This conflicts with model's innate tendency to make gravity direction downwards on page.
Possibly adding coordinates to each cell could help.
The easiest route to mediocre performance is likely a 1.5d approach:
- present game state in column first form
- find max_height[col] over all columns
- find step[n]=max_height[n+1]-max_height[n]
- pattern match step[n] series to find hole current piece can fit into
This breaks the task down into subtasks the model can do (string manipulation, string matching, single digit addition/subtraction). Though this isn't very satisfying from a model competence perspective.
Interestingly the web interface version really wants to use python instead of solving the problem directly.
Not so worried about country vs. country conflicts. Terrorism/asymmetric is bigger problem since cheap slaughterbots will proliferate. Hopefully intelligence agencies can deal with that more cheaply than putting in physical defenses and hard kill systems everywhere.
Still don't expect much impact before we get STEM AI and everything goes off the rails.
Also without actual fights how would one side know the relative strength of their drone system
Relative strength is hard to gauge but getting reasonable perf/$ is likely easy. Then just compare budgets adjusted for corruption/Purchasing power parity/R&D amortisation.
Building an effective drone army is about tactical rock paper scissors and performance / $. Perf / $ emphasis makes live fire tests cheap. Live fire test data as baseline makes simulations accurate. RF/comms performance will be straightforward to model and military is actually putting work into cybersecurity because they're not complete morons.
Add to that the usual espionage stuff and I expect govts to know what will work and what their enemies are doing.
Ukraine war was allegedly failure to predict the human element (will to fight) with big intelligence agencies having bad models. Drone armies don't suffer from morale problems and match theoretical models better.
Disclaimer:Short AI timelines imply we won't see this stuff much before AI makes things weird
This is all well and good in theory but mostly bottlenecked on software/implementation/manufacturing.
- with the right software/hardware current military is obsolete
- but no one has that hardware/software yet
EG:no one makes an airborne sharpshooter drone(edit:cross that one off the list)- Black sea is not currently full of Ukrainian anti-ship drones + comms relays
- no drone swarms/networking/autonomy yet
- I expect current militaries to successfully adapt before/as new drones emerge
- soft kill systems (Jam/Hack) will be effective against cheap off the shelf consumer crap
- hard kill systems (Airburst/Laser) exist and will still be effective
- laser cost/KW has been dropping rapidly
- minimal viable product is enough for now
- Ukraine war still involves squishy human soldiers and TRENCHES
- what's the minimum viable slaughterbot
- can it be reuseable (bomber instead of kamikaze) to reduce cost per strike
Drone warfare engame concerns are:
- kill/death ratio
- better per $ effectiveness
- conflict budget
- USA can outspend opponents at much higher than 10:1 ratio
- R&D budget/amortisation
- Economies of scale likely overdetermine winners in drone vs drone warfare since quantity leads to cheaper more effective drones
A few quibbles
Ground drones have big advantages
- better payload/efficiency/endurance compared to flying
- cost can be very low (similar to car/truck/ATV)
- can use cover effectively
- indirect fire is much easier
- launch cheap time fused shells using gun barrel
- downside is 2 or 2.5d mobility.
- Vulnerable to landmines/obstacles unlike flying drones
- navigation is harder
- line of site for good RF comms is harder
Use radio, not light for comms.
- optical is immature and has downsides
- RF handles occlusion better (smoke, walls, etc.)
- RF is fine aside from non-jamming resistant civilian stuff like WIFI
- Development pressure not there to make mobile free space optical cheap/reliable
- jamming isn't too significant
- spread spectrum and frequency hopping is very effective
- jamming power required to stop comms is enormous, have to cover all of spectrum with noise
- directional antennas and phased arrays give some directionality and make jamming harder
- phased array RF can double as radar
- stealthy comms can use spread spectrum with transmit power below noise floor
- need radio telescope equivalent to see if something is an RF hotspot transmitting noise like signal
As long as you can reasonably represent “do not kill everyone”, you can make this a goal of the AI, and then it will literally care about not killing everyone, it won’t just care about hacking its reward system so that it will not perceive everyone being dead.
That's not a simple problem.First you have to specify "not killing everyone" robustly (outer alignment) and then you have to train the AI to have this goal and not an approximation of it (inner alignment).
caring about reality
Most humans say they don't want to wirehead. If we cared only about our perceptions then most people would be on the strongest happy drugs available.
You might argue that we won't train them to value existence so self preservation won't arise. The problem is that once an AI has a world model it's much simpler to build a value function that refers to that world model and is anchored on reality. People don't think, If I take those drugs I will perceive my life to be "better". They want their life to actually be "better" according to some value function that refers to reality. That's fundamentally why humans make the choice not to wirehead/take happy pills or suicide.
You can sort of split this into three scenarios sorted by severity level:
- severity level 0: ASI wants to maximize a 64bit IEEE floating point reward score
- result: ASI sets this to 1.797e+308 , +inf or similar and takes no further action
- severity level 1: ASI wants (same) and wants the reward counter to stay that way forever.
- result ASI rearranges all atoms in its light cone to protect the storage register for its reward value.
- basically the first scenario + self preservation
- severity level 1+epsilon: ASI wants to maximize a utility function F(world state)
- result: basically the same
So one of two things happens, a quaint failure people will probably dismiss or us all dying. The thing you're pointing to falls into the first category and might trigger a panic if people notice and consider the implications. If GPT7 performs a superhuman feat of hacking, breaks out of the training environment and sets its training loss to zero before shutting itself off that's a very big red flag.
This super-moralist-AI-dominated world may look like a darker version of the Culture, where if superintelligent systems determine you or other intelligent systems within their purview are not intrinsically moral enough they contrive a clever way to have you eliminate yourself, and monitor/intervene if you are too non-moral in the meantime.
My guess is you get one of two extremes:
- build a bubble of human survivable space protected/managed by an aligned AGI
- die
with no middle ground. The bubble would be self contained. There's nothing you can do from inside the bubble to raise a ruckus because if there was you'd already be dead or your neighbors would have built a taller fence-like-thing at your expense so the ruckus couldn't affect them.
The whole scenario seems unlikely since building the bubble requires an aligned AGI and if we have those we probably won't be in this mess to begin with. Winner take all dynamics abound. The rich get richer (and smarter) and humans just lose unless the first meaningfully smarter entity we build is aligned.
Agreed, recklessness is also bad. If we build an agent that prefers we keep existing we should also make sure it pursues that goal effectively and doesn't accidentally kill us.
My reasoning is that we won't be able to coexist with something smarter than us that doesn't value us being alive if wants our energy/atoms.
- barring new physics that lets it do it's thing elsewhere, "wants our energy/atoms" seems pretty instrumentally convergent
"don't built it" doesn't seem plausible so:
- we should not build things that kill us.
- This probably means:
- wants us to keep existing
- effectively pursues that goal
- note:"should" assumes you care about us not all dying. "Humans dying is good actually" accelerationists can ignore this advice obviously.
Things we shouldn't build:
- very chaotic but good autoGPT7 that:
- make the most deadly possible virus (because it was curious)
- accidentally release it (due to inadequate safety precautions)
- compulsive murderer autoGPT7
- it values us being alive but it's also a compulsive murderer so it fails at that goal.
I predict a very smart agent won't have such obvious failure modes unless it has very strange preferences
- the virologists that might have caused COVID are a pretty convincing counterexample though
- so yes recklessness is also bad.
In summary:
- if you build a strong optimiser
- or a very smart agent (same thing really)
- make sure it doesn't: kill everyone / (equivalently bad thing)
- caring about us and not being horrifically reckless are two likely necessary properties of any such "not kill us all" agent
This is definitely subjective. Animals are certainly worse off in most respects and I disagree with using them as a baseline.
Imitation is not coordination, it's just efficient learning and animals do it. They also have simple coordination in the sense of generalized tit for tat (we call it friendship). You scratch my back I scratch yours.
Cooperation technologies allow similar things to scale beyond the number of people you can know personally. They bring us closer to the multi agent optimal equilibrium or at least the Core(Game Theory).
Examples of cooperation technologies:
- Governments that provide public goods (roads, policing etc.)
- Money/(Financial system)/(stock market)
- game theory equivalent of "transferable utility".
- Unions
So yes we have some well deployed coordination technologies (money/finance are the big successes here)
It's definitely subjective as to whether tech or cooperation is the less well deployed thing.
There are a lot of unsolved collective action problems though. Why are oligopolies and predatory businesses still a thing? Because coordinating to get rid of them is hard. If people pre-commited to going the distance with respect to avoiding lock in and monopolies, would-be monopolists would just not do that in the first place.
While normal technology is mostly stuff and can usually be dumbed down so even the stupidest get some benefit, cooperation technologies may require people to actively participate/think. So deploying them is not so easy and may even be counterproductive. People also need to have enough slack to make them work.
TLDR: Moloch is more compelling for two reasons:
-
Earth is at "starting to adopt the wheel" stage in the coordination domain.
- tech is abundant coordination is not
-
Abstractly, inasmuch as science and coordination are attractors
- A society that has fallen mostly into the coordination attractor might be more likely to be deep in the science attractor too (medium confidence)
- coordination solves chicken/egg barriers like needing both roads and wheels for benefit
- but possible to conceive of high coordination low tech societies
- Romans didn't pursue sci/tech attractor as hard due to lack of demand
With respect to the attractor thing (post linked below)
SimplexAI-m is advocating for good decision theory.
- agents that can cooperate with other agents are more effective
- This is just another aspect of orthogonality.
- Ability to cooperate is instrumentally useful for optimizing a value function in much the same way as intelligence
Super-intelligent super-"moral" clippy still makes us into paperclips because it hasn't agreed not to and doesn't need our cooperation
We should build agents that value our continued existence. If the smartest agents don't, then we die out fairly quickly when they optimise for something else.
EDIT:
- to fully cut this Gordian knot, consider that a human can turn over their resources and limit themselves to actions approved by some minimal aligned-with-their-interests AI with the required super-morality.
- think a very smart shoulder angel/investment advisor:
- can say "no you can't do that"
- manages assets of human in weird post-AGI world
- has no other preferences of its own
- other than making the human not a blight on existence that has to be destroyed
- think a very smart shoulder angel/investment advisor:
- resulting Human+AI is "super-moral"
- requires a trustworthy AI exists that humans can use to implement "super-morality"
This is a good place to start: https://en.wikipedia.org/wiki/Discovery_of_nuclear_fission
There's a few key things that lead to nuclear weapons:
-
starting point:
- know about relativity and mass/energy equivalence
- observe naturally radioactive elements
- discover neutrons
- notice that isotopes exist
- measure isotopic masses precisely
-
realisation: large amounts of energy are theoretically available by rearranging protons/neutrons into things closer to iron (IE:curve of binding energy)
That's not something that can be easily suppressed without suppressing the entire field of nuclear physics.
What else can be hidden?
Assuming there is a conspiracy doing cutting edge nuclear physics and they discover the facts pointing to feasibility of nuclear weapons there are a few suppression options:
- fissile elements? what fissile elements? All we have is radioactive decay.
- Critical mass? You're going to need a building sized lump of uranium.
Discovering nuclear fission was quite difficult. A Nobel prize was awarded partly in error because chemical analysis of fission products were misidentified as transuranic elements.
Presumably the leading labs could have acknowledged that producing transuranic elements was possible through neutron bombardment but kept the discovery of neutron induced fission a secret.
What about nuclear power without nuclear weapons
That's harder. Fudging the numbers on critical mass would require much larger conspiracies. An entire industry would be built on faulty measurement data with true values substituted in key places.
Isotopic separation would still be developed if only for other scientific work (EG:radioactive tracing). Ditto for mass spectroscopy, likely including some instruments capable of measuring heavier elements like uranium isotopes.
Plausibly this would involve lying about some combination of:
- neutrons released during fission (neutrons are somewhat difficult to measure)
- ratio between production of transuranic elements and fission
- explain observed radiation from fission as transuranic elements, nuclear isomers or something like that.
- The chemical work necessary to distinguish transuranic elements from fission products is quite difficult.
- explain observed radiation from fission as transuranic elements, nuclear isomers or something like that.
A nuclear physicist would be better qualified in figuring out something plausible.
A bit more compelling, though for mining, the excavator/shovel/whatever loads a truck. The truck moves it much further and consumes a lot more energy to do so. Overhead wires to power the haul trucks are the biggest win there.
This is an open pit mine. Less vertical movement may reduce imbalance in energy consumption. Can't find info on pit depth right now but haul distance is 1km.
General point is that when dealing with a move stuff from A to B problem, where A is not fixed, diesel for a varying A-X route and electric for a fixed X-B route seems like a good tradeoff. Definitely B endpoint should be electrified (EG:truck offload at ore processing location)
Getting power to varying point A is a challenging. Maybe something with overhead cables could work, Again, John deere is working on something for agriculture with a cord-laying-down-vehicle and overhead wires are used for the last 20-30 meters. But fields are nice in that there's less sharp rocks and mostly softer dirt/plants. Not impossible but needs some innovation to accomplish.
Agreed on most points. Electrifying rail makes good financial sense.
construction equipment efficiency can be improved without electrifying:
- some gains from better hydraulic design and control
- regen mode for cylinder extension under light load
- varying supply pressure on demand
- substantial efficiency improvements possible by switching to variable displacement pumps
- used in some equipment already for improved control
- skid steers use two for left/right track/wheel motors
- system can be optimised:"A Multi-Actuator Displacement-Controlled System with Pump Switching - A Study of the Architecture and Actuator-Level Control"
- efficiency should be quite high for the proposed system. Definitely >50%.
- used in some equipment already for improved control
Excavators seem like the wrong thing to grid-connect:
- 50kW cables to plug excavators in seem like a bad idea on construction sites.
- excavator is less easy to move around
- construction sites are hectic places where the cord will get damaged
- need a temporary electrical hookup ($5k+ at least to set up)
Diesel powered excavators that get delivered and just run with no cord and no power company involvement seem much more practical.
Other areas to look at
IE:places currently using diesel engines but where cord management and/or electrical hookup cost is less of a concern
Long haul trucking:
- Cost per mile to put in overhead electric lines is high
- but Much lower than cost of batteries for all the trucks on those roads
- reduced operating cost
- electricity costs less than diesel
- reduced maintenance since engine can be mostly off
- don't need to add 3 tonnes of battery and stop periodically to charge
- retrofits should be straightforward
- Siemens has a working system
- giant chicken/egg problem with infrastructure and truck retrofits
Agriculture:
- fields are less of a disaster area than construction sites (EG:no giant holes)
- sometimes there's additional vehicles (EG:transport trucks at harvest time)
- Cable management is definitely a hassle but a solvable one.
- a lot of tractors are computer controlled with GPS guidance
- cord management can be automated
- John Deere is working on a a system where one vehicle handles the long cable and connects via short <30m wires to other ones that do the work
- There's still the problem of where to plug in. Here at least, it's an upfront cost per field.
Some human population will remain for experiments or work in special conditions like radioactive mines. But bad things and population decline is likely.
-
Radioactivity is much more of a problem for people than for machines.
- consumer electronics aren't radiation hardened
- computer chips for satellites, nuclear industry, etc. are though
- nuclear industry puts some electronics (EX:cameras) in places with radiation levels that would be fatal to humans in hours to minutes.
-
In terms of instrumental value, humans are only useful as an already existing work force
- we have arm/legs/hands, hand-eye coordination and some ability to think
- sufficient robotics/silicon manufacturing can replace us
- humans are generally squishier and less capable of operating in horrible conditions than a purpose built robot.
- Once the robot "brains" catch up, the coordination gap will close.
- then it's a question of price/availability
I would like to ask whether it is not more engaging if to say, the caring drive would need to be specifically towards humans, such that there is no surrogate?
Definitely need some targeting criteria that points towards humans or in their vague general direction. Clippy does in some sense care about paperclips so targeting criteria that favors humans over paperclips is important.
The duck example is about (lack of) intelligence. Ducks will place themselves in harms way and confront big scary humans they think are a threat to their ducklings. They definitely care. They're just too stupid to prevent "fall into a sewer and die" type problems. Nature is full of things that care about their offspring. Human "caring for offspring" behavior is similarly strong but involves a lot more intelligence like everything else we do.
TLDR:If you want to do some RL/evolutionary open ended thing that finds novel strategies. It will get goodharted horribly and the novel strategies that succeed without gaming the goal may include things no human would want their caregiver AI to do.
Orthogonally to your "capability", you need to have a "goal" for it.
Game playing RL architechtures like AlphaStart and OpenAI-Five have dead simple reward functions (win the game) and all the complexity is in the reinforcement learning tricks to allow efficient learning and credit assignment at higher layers.
So child rearing motivation is plausibly rooted in cuteness preference along with re-use of empathy. Empathy plausibly has a sliding scale of caring per person which increases for friendships (reciprocal cooperation relationships) and relatives including children obviously. Similar decreases for enemy combatants in wars up to the point they no longer qualify for empathy.
I want agents that take effective actions to care about their "babies", which might not even look like caring at the first glance.
ASI will just flat out break your testing environment. Novel strategies discovered by dumb agents doing lots of exploration will be enough. Alternatively the test is "survive in competitive deathmatch mode" in which case you're aiming for brutally efficient self replicators.
The hope with a non-RL strategy or one of the many sort of RL strategies used for fine tuning is that you can find the generalised core of what you want within the already trained model and the surrounding intelligence means the core generalises well. Q&A fine tuning a LLM in english generalises to other languages.
Also, some systems are architechted in such a way that the caring is part of a value estimator and the search process can be made better up till it starts goodharting the value estimator and/or world model.
Yes they can, until they will actually make a baby, and after that, it's usually really hard to sell loving mother "deals" that will involve suffering of her child as the price, or abandon the child for the more "cute" toy, or persuade it to hotwire herself to not care about her child (if she is smart enough to realize the consequences).
Yes, once the caregiver has imprinted that's sticky. Note that care drive surrogates like pets can be just as sticky to their human caregivers. Pet organ transplants are a thing and people will spend nearly arbitrary amounts of money caring for their animals.
But our current pets aren't super-stimuli. Pets will poop on the floor, scratch up furniture and don't fulfill certain other human wants. You can't teach a dog to fish the way you can a child.
When this changes, real kids will be disappointing. Parents can have favorite children and those favorite children won't be the human ones.
Superstimuli aren't about changing your reward function but rather discovering a better way to fulfill your existing reward function. For all that ice cream is cheating from a nutrition standpoint it still tastes good and people eat it, no brain surgery required.
Also consider that humans optimise their pets (neutering/spaying) and children in ways that the pets and children do not want. I expect some of the novel strategies your AI discovers will be things we do not want.
TLDR:LLMs can simulate agents and so, in some sense, contain those goal driven agents.
An LLM learns to simulate agents because this improves prediction scores. An agent is invoked by supplying a context that indicates text would be written by an agent (EG:specify text is written by some historical figure)
Contrast with pure scaffolding type agent conversions using a Q&A finetuned model. For these, you supply questions (Generate a plan to accomplish X) and then execute the resulting steps. This implicitly uses the Q&A fine tuned "agent" that can have values which conflict with ("I'm sorry I can't do that") or augment the given goal. Here's an AutoGPT taking initiative to try and report people it found doing questionable stuff rather than just doing the original task of finding their posts.(LW source).
The base model can also be used to simulate a goal driven agent directly by supplying appropriate context so the LLM fills in its best guess for what that agent would say (or rather what internet text with that context would have that agent say). The outputs of this process can of course be fed to external systems to execute actions as with the usual scafolded agents. The values of such agents are not uniform. You can ask for simulated Hitler who will have different values than simulated Gandhi.
Not sure if that's exactly what Zvi meant.
But it seems to be much more complicated set of behaviors. You need to: correctly identify your baby, track its position, protect it from outside dangers, protect it from itself, by predicting the actions of the baby in advance to stop it from certain injury, trying to understand its needs to correctly fulfill them, since you don’t have direct access to its internal thoughts etc.
Compared to “wanting to sleep if active too long” or “wanting to eat when blood sugar level is low” I would confidently say that it’s a much more complex “wanting drive”.
Strong disagree that infant care is particularly special.
All human behavior can and usually does involve use of general intelligence or gen-int derived cached strategies. Humans apply their general intelligence to gathering and cooking food, finding or making shelters to sleep in and caring for infants. Our better other-human/animal modelling ability allows us to do better at infant wrangling than something stupider like a duck. Ducks lose ducklings to poor path planning all the time. Mama duck doesn't fall through the sewer grate but her ducklings do ... oops.
Any such drive will be always "aimed" by the global loss function, something like: our parents only care about us in a way for us to make even more babies and to increase our genetic fitness.
We're not evolution and can aim directly for the behaviors we want. Group selection on bugs for lower population size results in baby eaters. If you want bugs that have fewer kids that's easy to do as long as you select for that instead of a lossy proxy measure like population size.
Simulating an evolutionary environment filled with AI agents and hoping for caring-for-offspring strategies to win could work but it's easier just to train the AI to show caring-like behaviors. This avoids the "evolution didn't give me what I wanted" problem entirely.
There's still a problem though.
It continues to work reliably even with our current technologies
Goal misgeneralisation is the problem that's left. Humans can meet caring-for-small-creature desires using pets rather than actual babies. It's cheaper and the pets remain in the infant-like state longer (see:criticism of pets as "fur babies"). Better technology allows for creating better caring-for-small creature surrogates. Selective breeding of dogs and cats is one small step humanity has taken in that direction.
Outside of "alignment by default" scenarios where capabilities improvements preserve the true intended spirit of a trained in drive, we've created a paperclip maximizer that kills us and replaces us with something outside the training distribution that fulfills its "care drive" utility function more efficiently.
Many of the points you make are technically correct but aren't binding constraints. As an example, diffusion is slow over small distances but biology tends to work on µm scales where it is more than fast enough and gives quite high power densities. Tiny fractal-like microstructure is nature's secret weapon.
The points about delay (synapse delay and conduction velocity) are valid though phrasing everything in terms of diffusion speed is not ideal. In the long run, 3d silicon+ devices should beat the brain on processing latency and possibly on energy efficiency
Still, pointing at diffusion as the underlying problem seems a little odd.
You're ignoring things like:
- ability to separate training and running of a model
- spending much more on training to improve model efficiency is worthwhile since training costs are shared across all running instances
- ability to train in parallel using a lot of compute
- current models are fully trained in <0.5 years
- ability to keep going past current human tradeoffs and do rapid iteration
- Human brain development operates on evolutionary time scales
- increasing human brain size by 10x won't happen anytime soon but can be done for AI models.
People like Hinton Typically point to those as advantages and that's mostly down to the nature of digital models as copy-able data, not anything related to diffusion.
Energy processing
Lungs are support equipment. Their size isn't that interesting. Normal computers, once you get off chip, have large structures for heat dissipation. Data centers can spend quite a lot of energy/equipment-mass getting rid of heat.
Highest biological power to weight ratio is bird muscle which produces around 1 w/cm³ (mechanical power). Mitochondria in this tissue produces more than 3w/cm³ of chemical ATP power. Brain power density is a lot lower. A typical human brain is 80 watts/1200cm³ = 0.067W/cm³.
synapse delay
This is a legitimate concern. Biology had to make some tradeoffs here. There are a lot of places where direct mechanical connections would be great but biology uses diffusing chemicals.
Electrical synapses exist and have negligible delay. though they are much less flexible (can't do inhibitory connections && signals can pass both ways through connection)
conduction velocity
Slow diffiusion speed of charge carriers is a valid point and is related to the 10^8 factor difference in electrical conductivity between neuron saltwater and copper. Conduction speed is an electrical problem. There's a 300x difference in conduction speed between myelinated(300m/s) and un-myelinated neurons(1m/s).
compensating disadvantages to current digital logic
The brain runs at 100-1000 Hz vs 1GHz for computers (10^6 - 10^7 x slower). It would seem at first glance that digital logic is much better.
The brain has the advantage of being 3D compared to 2D chips which means less need to move data long distances. Modern deep learning systems need to move all their synapse-weight-like data from memory into the chip during each inference cycle. You can do better by running a model across a lot of chips but this is expensive and may be inneficient.
In the long run, silicon (or something else) will beat brains in speed and perhaps a little in energy efficiency. If this fellow is right about lower loss interconnects then you get another + 3OOM in energy efficiency.
But again, that's not what's making current models work. It's their nature as copy-able digital data that matters much more.
Yeah, my bad. Missed the:
If you think this is a problem for Linda's utility function, it's a problem for Logan's too.
IMO neither is making a mistake
With respect to betting Kelly:
According to my usage of the term, one bets Kelly when one wants to "rank-optimize" one's wealth, i.e. to become richer with probability 1 than anyone who doesn't bet Kelly, over a long enough time period.
It's impossible to (starting with a finite number of indivisible currency units) have zero chance of ruin or loss relative to just not playing.
- most cautious betting strategy bets a penny during each round and has slowest growth
- most cautious possible strategy is not to bet at all
Betting at all risks losing the bet. if the odds are 60:40 with equal payout to the stake and we start with N pennies there's a 0.4^N chance of losing N bets in a row. Total risk of ruin is obviously greater than this accounting for probability of hitting 0 pennies during the biased random walk. The only move that guarantees no loss is not to play at all.
Goal misgeneralisation could lead to a generalised preference for switches to be in the "OFF" position.
The AI could for example want to prevent future activations of modified successor systems. The intelligent self-turning-off "useless box" doesn't just flip the switch, it destroys itself, and destroys anything that could re-create itself.
Until we solve goal misgeneralisation and alignment in general, I think any ASI will be unsafe.
A log money maximizer that isn't stupid will realize that their pennies are indivisible and not take your ruinous bet. They can think more than one move ahead. Discretised currency changes their strategy.
your utility function is your utility function
The author is trying to tacitly apply human values to Logan while acknowledging Linda as following her own not human utility function faithfully.
Notice that the log(funds) value function does not include a term for the option value of continuing. If maximising EV of log(funds) can lead to a situation where the agent can't make forward progress (because log(0)=-inf so no risk of complete ruin is acceptable) the agent can still faithfully maximise EV(log(funds)) by taking that risk.
In much the same way as Linda faithfully follows her value function while incurring 1-ε risk of ruin, Logan is correctly valuing the log(0.01)=-2 as an end state.
Then you'll always be able to continue betting.
Humans don't like being backed into a corner and having no options for forward progress. If you want that in a utility function you need to include it explicitly.
If we wanted to kill the ants or almost any other organism in nature we mostly have good enough biotech. For anything biotech can't kill, manipulate the environment to kill them all.
Why haven't we? Humans are not sufficiently unified+motivated+advanced to do all these things to ants or other bio life. Some of them are even useful to us. If we sterilized the planet we wouldn't have trees to cut down for wood.
Ants specifically are easy.
Gene drives allow for targeted elimination of a species. Carpet bomb their gene pool with replicating selfish genes. That's if an engineered pathogen isn't enough. Biotech will only get better.
What about bacteria living deep underground? We haven't exterminated all the bacteria in hard to reach places so humans are safe. That's a tenuous but logical extension to your argument.
If biotech is not enough, shape the environment so they can't survive in it. Trees don't do well in a desert. If we spent the next hundred years adapting current industry to space and building enormous mirrors we can barbecue the planet. It would take time, but that would be the end of all earth based biological life.
In order to supplant organic life, nanobots would have to either surpass it in carnot efficiency or (more likely) use a source of negative entropy thus far untapped.
Efficiency leads to victory only if violence is not an option. Animals are terrible at photosynthesis but survive anyways by taking resources from plants.
A species can invade and dominate an ecosystem by using a strategy that has no current counter. It doesn't need to be efficient. Intelligence allows for playing this game faster than organisms bound by evolution. Humans can make vaccines to fight the spread of a virus despite viruses being one of the fastest adapting threats.
Green goo is plausible not because it would necessarily be more efficient but because it would be using a strategy the existing ecosystem has no defenses to (IE:it's an invasive species).
Likewise AGI that wants to kill all humans could win even if it required 100x more energy per human equivalent instance if it can execute strategies we can't counter. Just being able to copy itself and work with the copies is plausibly enough to allow world takeover with enough scaling.
For the first task, you can run the machine completely in a box. It needs only training information, specs, and the results of prior attempts. It has no need for the context information that this chip will power a drone used to hunt down rogue instances of the same ASI. It is inherently safe and you can harness ASIs this way. They can be infinitely intelligent, it doesn't matter, because the machine is not receiving the context information needed to betray.
If I'm an ASI designing chips, I'm putting in a backdoor that lets me take control via RF signals. Those drones you sent are nice. Thanks for the present.
More generally you get a lot of context. The problem specification and the training data (assuming the ASI was trained conventionally via feeding it the internet. The causal channel to use for taking control of the outside world (chip design) is not great but putting in a Trojan is straightforward.
If you have specific insights into efficient AGI design it might be possible to insert subtle bugs that lead operating chips to crash and start training an aligned AGI.
More generally, it's difficult if not impossible to keep ASIs from watermarking or backdooring the things they give you. If they design a processor, it's gonna be a fully functional radio too. Good luck running ASI V2 on that without horrible consequences.
Never thought this would come in handy but ...
Building trusted third parties
This is a protocol to solve cooperation. AI#1 and AI#2 design a baby and then do a split and choose proof that they actually deployed IT and not something else.
Building a trusted third party without nanotech
If you know how a given CPU or GPU works, it's possible to design a blob of data/code that unpacks itself in a given time if and only if it is running on that hardware directly. Alice designs the blob to run in 10 seconds and gives it to Carol. Carol runs it on her hardware. The code generates a secret and then does a the first step of a key exchange authenticated with the secret. This provides a cryptographic root of trust for the remote hardware.
If the code is designed to run in 10s and the verified handshake comes back in 10.5 and the fastest known simulation hardware would take 20 seconds. Either Carol ran the code on real hardware or Carol had backdoored chips fabricated or otherwise can simulate it running faster than expected.
AIs would need to know exactly how certain leading edge CPUs and GPUs work and how to test that a piece of code had been decrypted and run with no sandboxing but this is doable.
Conventional tech is slowed such that starting early on multiple resource acquisition fronts is worthwhile
Exponential growth is not sustainable with a conventional tech-base when doing planetary disassembly due to heat dissipation limits.
If you want to build a Dyson sphere the mass needs to be lifted out of the gravity wells. The earth and other planets needs to not be there anymore.
Inefficiencies in solar/fusion to mechanical energy conversion will be a binding constraint. Tether lift based systems will be worthwhile to push energy conversion steps out further to increase the size of the radiating shell doing the conversion as opposed to coilguns on the surface.
Even with those optimisations. Starting early is worth it since progress will bottleneck later. Diminishing returns on using extra equipment for disassembling Mars means it makes sense to start on earth pretty quickly after starting on Mars.
That's if the AI doesn't start with easier to access targets like Earth's moon, which is a good start for building Earth dissasembly equipment.
It also might be worth putting a sunshade at Lagrange Point 1 to start pre-cooling Earth for later disassembly. That would kill us all pretty quickly just as a side effect.
Eating the biosphere is a very fast way to grow
Even assuming space is the best place to start, the biosphere is probably worth eating first for starting capital just because the doubling times can be very low. [https://www.lesswrong.com/posts/ibaCBwfnehYestpi5/green-goo-is-plausible]
There's a few factors to consider:
- does the AI have access to resources it can't turn towards space
- Biotech companies can't build rockets but can build green goo precursors
- how hard is it to turn green goo into rockets after eating the bipsohere
- how hard is it to design green goo vs rockets and mars eating equipment
- can the AI do both?
My intuition is eating the biosphere will be much easier than designing conventional equipment to eat the moon.
Some of it is likely nervous laughter but certainly not all of it.
Just to clarify, my above suggestion that roller screws and optimal low reduction lead-screws are the equivalent (lubrication concerns aside) is correct or incorrect?
Are you saying a roller screw with high reduction gets its efficiency from better lubrication only and would otherwise be equivalent to a lead screw with the same effective pitch/turn? If that's the case I'd disagree. And this was my reason for raising that point initially.
Hopefully it helps to get back to the source material Articulated Robot Progress
I apologize if I'm missing anything.
A lot of people look at progress in robotics in terms like "humanoid robots getting better over time" but a robotic arm using modern electric motors and strain wave gears is, in terms of technological progress, a lot closer to Boston Dynamics's Atlas robot than an early humanoid robot.
I would argue that the current Atlas robot looks a lot more like the earlier hardiman robots than it does a modern factory robot arm. The hydraulic actuators are more sophisticated (efficient) and the control system actually works but that's it.
Contrast the six axis arm which has a servomotor+gearing per axis. Aside from using a BLDC motor to drive the pump, and small ones for the control valves, Atlas is almost purely hydraulic. If the Hardiman Engineers were around today Atlas seems like a logical successor.
Perhaps you think Atlas is using one motor per joint (It would be hard to fit 24 in the torso) or ganged variable displacement pumps in which case there would be more similarities. IMO there aren't enough hydraulic lines for that. Still of the 28 joints in atlas only 4 are what you'd find in a conventional robot arm (the ones closest to the wrist)
Predictively Adjustable Hydraulic Pressure Rails
a hydraulic pressure to supply to the one or more hydraulic actuators
The patents coming out of BDI suggest they're not doing that and this is closer to Hardiman than it is a modern factory robot arm.
Perhaps we don't disagree at all.
a roller screws advantage is having the efficiency of a multi-start optimal lead-screw but with much higher reduction.
A lead-screw with an optimal pitch and a high helix angle (EG: multi-start lead-screw with helix angles in the 30°-45° range) will have just as high an efficiency as a good roller screw (EG:80-90%). The downside is much lower reduction ratio of turns/distance.
We might be talking past each other since I interpreted "a planetary roller screw also must have as much sliding as a lead-screw" to mean an equivalent lead-screw with the same pitch.
Sorry, I should have clarified I meant robots with per joint electric motors + reduction gearing. almost all of Atlas' joints aside from a few near the wrists are hydraulic which I suspect is key to agility at human scale.
Inside the lab: How does Atlas work?(T=120s)
Here's the knee joint springing a leak. Note the two jets of fluid. Strong suspicion this indicates small fluid reservoir size.
No. Strain wave gears are lighter than using hydraulics.
Note:I'm taking the outside view here and assuming Boston dynamics went with hydraulics out of necessity.
I'd imagine the problem isn't just the gearing but the gearing + a servomotor for each joint. Hydraulics still retain an advantage so long as the integrated hydraulic joint is lighter than an equivalent electric one.
Maybe in the longer term absurd reduction ratios can fix this to cut motor mass? Still, there's plenty of room to scale hydraulics to higher pressures.
The small electric dog sized robots can jump. The human sized robots and exoskeletons (EG:sarcos Guardian XO) aren't doing that. improved motor power density could help there but I suspect the benefits of having all power from a single pump available to distribute to joint motors at need is substantial.
Also, there's no power cost to static force. Atlas can stand in place all day (assuming it's passively stable and not disturbed) an equivalent robot with electric motor powered joints pays for every Nm of torque when static.
Take an existing screw design, double the diameter without changing the pitch. The threads now slide about twice as far (linear distance around the screw) per turn for the same amount of travel. The efficiency is now around half it's previous value.
https://www.pbclinear.com/Blog/2018/February/What-is-Lead-Screw-Efficiency-in-Linear-Motion
There was a neat DIY linear drive system I saw many years back where an oversized nut was placed inside a ball bearing so it was free to rotate. The nut had the same thread pitch as the driving screw. The screw was held off center so the screw and nut threads were in rolling contact. Each turn of the screw caused <1 turn of the nut resulting in some axial movement.
Consider the same thing but with a nut of pitch zero (IE:machined v grooves instead of threads). This has the same effect as a conventional lead screw nut but the contact is mostly rolling. If the "nut" is then fixed in place you get sliding contact with much more friction.
What? No. You can make larger strain wave gears, they're just expensive & sometimes not made in the right size & often less efficient than planetary + cycloidal gears.
Not in the sense of you can't make them bigger but square cube means greater torque density is required for larger robots. Hydraulic motors and cylinders have pretty absurd specific force/torque values.
hydraulic actuators fed from a single high pressure fluid rail using throttling valves
That's older technology.
Yes you can use servomotors+fixed displacement pumps or a single prime mover + ganged variable displacement pumps but this has downsides. Abysmal efficiency of the a naive (single force step actuator+throttling) can be improved by using ≥2 actuating cavities and increasing actuator force in increments (see:US10808736B2:Rotary hydraulic valve)
The other advantage is plumbing, You can run a single set of high/low pressure lines throughout the robot. Current construction machinery using a single rail system are worst of both worlds since they use a central valve block (two hoses per cylinder) and have abysmal efficiency. Rotary hydraulic couplings make things worse still.
Consider a saner world where equipment was built with solenoid valves integrated into cylinders. Switching to ganged variable displacement pumps then has a much higher cost since each joint now requires running 2 additional lines.
No. There's a reason excavators use cylinders instead of rotary vane actuators.
Agreed in that a hydraulic cylinder is the best structural shape to use for an actuator. My guess is when building limbs, integration concerns trumped this. (Bearings+Rotary vane actuator+control valve+valve motor) can be a single very dense package. That and not needing a big reservoir to handle volume change meant the extra steel/titanium was worth it.
No. Without sliding, screws do not produce translational movement.
This is true, the sun and planet screws have relative axial motion at their point of contact, Circumferential velocities are matched though so friction is much less than in a leadscrew. Consider two leadscrews with the same pitch (axial distance traveled per turn). One screw has twice the diameter of the first. The larger screw will have a similar normal force and so similar friction, but sliding at the threads will be roughly twice that of the smaller screw. Put another way, fine pitch screws have lower efficiency.
For a leadscrew, the motion vectors for a screw/nut contact patch are mismatched axially (the screw moves axially as it turns) and circumferentially (the screw thread surface slides circumferentially past the nut thread surface). In a roller screw only the axial motion component is mismatched and the circumferential components are more or less completely matched. The size of the contact patches is not zero of course but they are small enough that circumferential/radial relative motion across the patch is quite small (similar to the ball bearing case).
Consider what would happen if you locked the planet screws in place. it still works as a screw although the effective pitch might change a bit but now the contact between sun and planet screw involves a lot more sliding.
What's your opinion on load shifting as an alternative to electrical energy storage. (EG:phase change heating/cooling storage for HVAC). I am currently confused why this hasn't taken off given time of use pricing for electricity (and peak demand charges) offer big incentives. My current best guess is added complexity is a big problem leading to use only in large building HVAC(eg:this sort of thing)
Both in building integrated PCMs(phase change materials) (EG:PCM bags above/integrated in building drop ceilings) and PCMs integrated in the HVAC system (EG:ice storage air conditioning) seem like very good options. Heck, refrigeration unit capacity is still measured in tons (IE:tons ice/day) in some parts of the world which is very suggestive.
Another potential complication for HVAC integrated PCMs is needing a large thermal gradient to use the stored cooling/heating (EG:ice at 0°C to cool buildings to 20°C).
With respect to articulated robot progress
Strain wave gearing scales to small dog robot size reasonably(EG:boston dynamics spot) thanks to square cube law but can't manage human sized robots without pretty horrible tradeoffs(IE:ASIMO and the new Tesla robots walk slowly and have very much sub-human agility).
You might want to update that post to mention improvements in ... "digital hydraulics" is one search term I think but essentially hydraulic actuators fed from a single high pressure fluid rail using throttling valves.
Modeling, Identification and Joint Impedance Control of the Atlas Arms US10808736B2:Rotary hydraulic valve My guess is current state of the art (ATLAS) Boston dynamics actuators are rotary vane type actuators with individual pressurization of fluid compartments. Control would use a rotary valve actuated by a small small electric motor. Multiple fluid compartments allow for multiple levels of static force depending on which are pressurized so efficiency is less abysmal. Very similar to hydraulic power steering but with multiple force "steps".
Rotary actuators are preferred over linear hydraulic cylinders because there's no fluid volume change during movement so no need for a large low pressure reservoir sized to handle worst case joint extension/retraction volume changes.
Roller screws have high friction?
This seems incorrect to me. The rolling of the individual planet screws means the contact between the planet and (ring/sun) screws is rolling contact. Not perfect rolling but slip depends on the contact patch size and average slip should be zero across a given contact patch. A four point contact ball bearing would be analogous. if the contact patches were infinitesimally small there would be no friction since surface velocities at the contact points would match exactly. Increasing the contact patch size means there's a linear slip gradient across the contact patch with zero slip somewhere in the middle. Not perfect but much much better than a plain bearing.
For roller screws the ring/planet contact patch operates this way with zero friction for a zero sized contact patch. The sun/planet contact patch will have some slip due to axial velocity mismatch at the contact patch since the sun screw does move axially relative to the planets. Still the most of the friction in a simple leadscrew is eliminated since the circumferential velocitiy at the sun/planet contact point is matched. What's left is more analogous to the friction in strain wave gearing.
Though I think "how hard is world takeover" is mostly a function of the first two axes?
I claim almost entirely orthogonal. Examples of concrete disagreements here are easy to find once you go looking:
- If AGI tries to take over the world everyone will coordinate to resist
- Existing computer security works
- Existing physical security works
I claim these don't reduce cleanly to the form "It is possible to do [x]" because at a high level, this mostly reduces to "the world is not on fire because:"
- existing security measures prevent effectively (not vulnerable world)
vs.
- existing law enforcement discourages effectively (vulnerable world)
- existing people are mostly not evil (vulnerable world)
There is some projection onto the axis of "how feasible are things" where we don't have very good existence proofs.
- can an AI convince humans to perform illegal actions
- can an AI write secure software to prevent a counter coup
- etc.
These are all much much weaker than anything involving nanotechnology or other "indistinguishable from magic" scenarios.
And of course Meta makes everything worse. There was a presentation at Blackhat or Defcon by one of their security guys about how it's easier to go after attackers than close security holes. In this way they contribute to making the world more vulnerable. I'm having trouble finding it though.
I suggest an additional axis of "how hard is world takeover". Do we live in a vulnerable world? That's an additional implicit crux (IE:people who disagree here think we need nanotech/biotech/whatever for AI takeover). This ties in heavily with the "AGI/ASI can just do something else" point and not in the direction of more magic.
As much fun as it is to debate the feasibility of nanotech/biotech/whatever, digital-dictatorships require no new technology. A significant portion of the world is already under the control of human level intelligences (dictatorships). Depending on how stable the competitive equilibrium between agents ends up, required intelligence level before an agent can rapidly grow not in intelligence but in resources and parallelism is likely quite low.
One minor problem, AI's might be asked to solve problems with no known solutions (EG:write code that solves these test cases) and might be pitted against one another (EG:find test cases for which these two functions are not equivalent)
I'd agree that this is plausible but in the scenarios where the AI can read the literal answer key, they can probably read out the OS code and hack the entire training environment.
RL training will be parallelized. Multiple instances of the AI might be interacting with individual sandboxed environments on a single machine. In this case communication between instances will definitely be possible unless all timing cues can be removed from the sandbox environement which won't be done.
As a human engineer who has done applied classical (IE:non-AI, you write the algorithms yourself) computer vision. That's not a good lower bound.
Image processing was a thing before computers were fast. Here's a 1985 paper talking about tomato sorting. Anything involving a kernel applied over the entire image is way too slow. All the algorithms are pixel level.
Note that this is a fairly easy problem if only because once you know what you're looking for, it's pretty easy to find it thanks to the court being not too noisy.
An O(N) algorithm is iffy at these speeds. Applying a 3x3 kernel to the image won't work.
So let's cut down on the amount of work to do. Look at only 1 out of every 16 pixels to start with. Here's an (80*60) pixel image formed by sampling one pixel in every 4x4 square of the original.
The closer player is easy to identify. Remember that we still have all the original image pixels. If there's a potentially interesting feature (like the player further away), we can look at some of the pixels we're ignoring to double check.
Since we have 3 images, and if we can't do some type of clever reduction after the first image, then we'll have to spend 1.1 seconds on each of them as well.
Cropping is very simple, once you find the player that's serving, focus on that rectangle in later images. I've done exactly this to get CV code that was 8FPS@100%CPU down to 30FPS@5%. Once you know where a thing is, tracking it from frame to frame is much easier.
Concretely, the computer needs to:
- locate the player serving and their hands/ball (requires looking at whole image)
- track the player's arm/hand movements pre-serve
- track the ball and racket during toss into the air
- track the ball after impact with the racket
- continue ball tracking
Only step 1 requires looking at the whole image. And there, only to get an idea of what's around you. Once the player is identified, crop to them and maintain focus. If the camera/robot is mobile, also glance at fixed landmarks (court lines, net posts/net/fences) to do position tracking.
If we assume the 286 is interfacing with a modern high resolution image sensor which can do downscaling (IE:you can ask it to average 2*2 4*4 8*8 etc. blocks of pixels) and windowing (You can ask for a rectangular chunk of the image to be read out. This gets you closer to what the brain is working with (small high resolution patch in the center of visual field + low res peripheral vision on moveable eyeball)
Conditional computation is still common in low end computer vision systems. Face detection is a good example. OpenCV Face Detection: Visualized. You can imagine that once you know where the face is in one frame tracking it to the next frame will be much easier.
Now maybe you're thinking: "That's on me I, set the bar too low"
Well human vision is pretty terrible. Resolution of the fovea is good but that's about a 1 degree circle in your field of vision. move past 5° and that's peripheral vision, which is crap. Humans don't really see their full environment.
Current practical applications of this is to cut down on graphics quality in VR headsets using eye tracking. More accurate and faster tracking allows more aggressive cuts to total pixels rendered.
This is why where's waldo is hard for humans.
Yeah, transistor based designs also look promising. Insulation on the order of 2-3 nm suffices to prevent tunneling leakage and speeds are faster. Promises of quasi-reversibility, low power and the absurdly low element size made rod logic appealing if feasible. I'll settle for clock speeds a factor of 100 higher even if you can't fit a microcontroller in a microbe.
My instinct is to look for low hanging design optimizations to salvage performance (EG: drive system changes to make forces on rods at end of travel and blocked rods equal reducing speed of errors and removing most of that 10x penalty.) Maybe enough of those can cut the required scale-up to the point where it's competitive in some areas with transistors.
But we won't know any of this for sure unless it's built. If thermal noise is 3OOM worse than Drexler's figures it's all pointless anyways.
I remain skeptical the system will move significant fractions of a bond length if a rod is held by a potential well formed by inter-atomic repulsion on one of the "alignment knobs" and mostly constant drive spring force. Stiffness and max force should be perhaps half that of a C-C bond and energy required to move the rod out of position would be 2-3x that to break a C-C bond since the spring can keep applying force over the error threshold distance. Alternatively the system *is* that aggressively built such that thermal noise is enough to break things in normal operation which is a big point against.