Posts
Comments
Am I still eligible for the prize if I publish a public blog post at the same time I submit the Google Doc or would you prefer I not publish a blog post about February 15th? Publishing the blog post immediately advances science better (because it can enable discussion) but waiting until after the February 15th might be preferable to you for contest-related reasons.
Whoops. Fixed. Thanks.
They were typos too. Fixed. Thanks.
Thanks. Fixed the first typo. Clarified the other bit.
Only eating with a fork. A full system would require more data than that. We tested on real people in real-world conditions who were not part of the training dataset. If someone ate in a different style we could add just a little bit of annotated training data for the eating style, run the toolchain overnight and the algorithm would be noticeably better for that person and everyone else. The reason why I'm so confident in our algorith was because ① it required very little data to do updates and ② I had lots of experience in the field which meant I knew exactly what quality level was and wasn't acceptable to customers.
To update the code in response to user feedback we would have to push the new code. Building an update system was theoretically straightforward. It was a (theoretically) solved problem with little technical risk. But it was not a problem that we had personally built a toolchain for and the whole firmware update system involved more technical maintenance than I wanted to commit myself to.
We should continue this conversation privately in either Less Wrong private messages or email.
Thanks. I also thought that Lsusr should look into this and try to replicate it.
Update: It gets higher if you run it for long enough.
testing 59300000: 0: 58.615 1: 57.36
testing 59320000: 0: 50.902 1: 50.28
testing 59340000: 0: 68.415 1: 66.67
testing 59360000: 0: 71.813 1: 69.36
testing 59380000: 0: 70.275 1: 68.53
testing 59400000: 0: 71.577 1: 68.67
Initial results indicate the code performs poorly on F-MNIST. It is possible this is a hyperparameter-tuning issue but my default conclusion is that MNIST (created in 1998, before the invention of modern GPUs) is just too easy.
testing 20000: 0: 44.413 1: 43.77
testing 40000: 0: 23.702 1: 23.34
testing 60000: 0: 21.822 1: 20.92
testing 80000: 0: 38.627 1: 37.75
testing 100000: 0: 25.107 1: 24.85
testing 120000: 0: 28.893 1: 28.34
testing 140000: 0: 29.203 1: 29.50
testing 160000: 0: 40.437 1: 39.55
testing 180000: 0: 49.828 1: 48.59
testing 200000: 0: 39.510 1: 39.81
testing 220000: 0: 39.938 1: 40.10
testing 240000: 0: 32.390 1: 31.65
testing 260000: 0: 25.367 1: 24.51
testing 280000: 0: 29.198 1: 28.66
testing 300000: 0: 29.823 1: 28.91
testing 320000: 0: 35.178 1: 34.20
testing 340000: 0: 32.370 1: 31.94
testing 360000: 0: 29.083 1: 28.31
testing 380000: 0: 30.117 1: 30.05
testing 400000: 0: 39.125 1: 39.21
I like this idea. It seems to me like a fair test. I will run the code overnight with default settings and see what happens.
You didn't miss it. Your quote and the later bit about "The question to ask is not ‘how’ to learn, but ‘when’." seem to contradict each other. I think your quote is just a general allegory to Hebb's rule and that it's not meant to be taken as a literal system spec, but I could be wrong. I am a confused by the original description.
Does MIRI do much in the way of capabilities research? It is my understanding that they don't. If MIRI doesn't do capabilities research then it seems unlikely to me they would do much with an idea that is all about capabilities advancement.
Here are the results after running the code for several hours on my CPUs.
testing 20000: 0: 77.553 1: 77.74
testing 40000: 0: 83.525 1: 83.33
testing 60000: 0: 87.880 1: 87.44
testing 80000: 0: 91.203 1: 90.10
⋮
testing 10860000: 0: 99.958 1: 98.50
testing 10880000: 0: 99.957 1: 98.47
testing 10900000: 0: 99.957 1: 98.56
testing 10920000: 0: 99.955 1: 98.47
D𝜋 recommends the following compiler flags to run the code on a GPU.
$ nvcc -ccbin g++ -I./cuda/NVIDIA_CUDA-9.2_Samples/common/inc -I/usr/local/cuda-9.2/targets/x86_64-linux/include -m64 -Xcompiler -fno-stack-protector -gencode arch=compute_61,code=sm_61 -gencode arch=compute_61,code=compute_61 -rdc=true -O2 sonn.o -c sonn.c
$ nvcc -ccbin g++ -m64 -Xcompiler -fno-stack-protector -gencode arch=compute_61,code=sm_61 -gencode arch=compute_61,code=compute_61 -o sonn sonn.o -lcuda
If you're using CUDA version 10.1 instead of 9.2 then you will have to rename sonn.c to sonn.cu.
I live near the UW. As far as I can tell, the UW has done a great job of pandemic response. I got a COVID test from them early in the pandemic before there were alternative tests available.
The LCSS algorithm (from which WarpingLCSS is an optimization) operates on a sequence of discrete symbols such as letters or nucleotides. This is in contrast to a neutral network whose natural inputs come from a continuous vector space. IMU data is continuous. We needed to bucket it before feeding it into LCSS. The random forest buckets continuous data into discrete categories. We tried support vector machines too but random forest classification worked better.
The code compiles. It runs faster than advertised (assuming I'm reading the output correctly). I guess that means I'm running on better hardware than D𝜋 used.
I'd rather not get into the details.
The website www.yann.lecun.com/ would not load on my machine. The website does load if I go to it without the www. prefix instead. Even more specifically, the place to get the raw data is this page.
Thanks! You are very welcome. I look forward to finding out what you'l be doing in five to ten years.
The Animatrix is related to the The Matrix. As far as I can tell, they all take place in the same universe. But you don't have to watch The Animatrix before you watch The Matrix Resurrections. You can watch The Animatrix before or after The Matrix Resurrections. Either way works just fine.
If you want to squeeze every possible drop of enjoyment out of The Matrix Resurrections then you should watch The Matrix Reloaded and The Matrix Revolutions before watching The Matrix Resurrections. However, I do not recommend that you actually watch The Matrix Reloaded and The Matrix Revolutions at all because The Matrix Reloaded and The Matrix Revolutions are (in my opinion) boring.
I have a narrower question. Why don't you, personally, make this happen?
When I first got vaccinated I signed up for a symptom reporting system that bugged me periodically to ask how I'm feeling. It seemed to me like the obvious way to monitor the population for vaccine effectiveness, vaccine side effects and symptomatic infection of the vaccinated population. When my second dose of the vaccine produced mild side effects I reported then via the automated survey.
A random walk-by testing survey station tested me (and anyone else who agreed) in a park. It seemed to me like a reasonable way to measure the relative ratio of asymptomatic variants.
For what it's worth, I took Valentine's first paragraph as high praise. I write narratives with the deliberate intention of eliciting specific emotional responses and that was the exact emotional response I aimed to elicit when I wrote this story.
I'd feel icky too to read such a response to one of your [Richard_Kennaway's] posts or to one of my drier posts. But I feel the emoting is appropriate in this context.
I don't recommend anyone use that actual literal hardware we used. Hardware advances fast and some of the components we used are no longer manufactured by Nordic Semiconductor. It would be better to start from scratch with new hardware. The hardware was not complicated. It was just an industry-standard IMU attached to an industry-standard microcontroller attached to a battery, a vibrating motor and a charging light.
If someone wants to take on this project the thing to steal from my experience would be the machine learning architecture. That's where all the hard technical challenges were. I think I have left behind enough hints in my story to save them 80% of the algorithmic work. Anyone competent enough to pull this project off could probably muddle through the remaining 20% on their own, but I recommend they hire me instead.
Possibly. I don't remember the name of their company.
Several years ago.
Your description of the invention is precisely correct. Cross posting on the EA forum is a good idea.
I'd enthusiastically do part-time contract work to help make a machine like this happen again, for the right price (which might be surprisingly high or low, depending on one's expectations). It'd be way faster and cheaper for a company to hire me than to do it themselves from scratch. It'd be less risky too. My mental model is that the bottleneck on a project like this is good founders/executives. It is a question of leadership and will.
My time is limited. I had more important things to do.
Not on an Apple Watch. At least, not on an Apple Watch the one time we tried porting our code to it. But it's theoretically possible to implement the algorithm on any wearable with an IMU which meets basic power efficiency requirements and which let's you run raw C close to the metal. Apple didn't let us run the app with the necessary privileges.
In the very beginning we wrote our software for Android smartwatches but that approach caused several problems.
- Our customers almost never had an Android smartwatch already. They bought whatever hardware we told them to. We earned $0 per sale even though Android smartwatches cost several times more money for our customers to buy. Customers kept asking us to make our own devices.
- Many of our customers were iPhone users. Android smartwatches do not integrate optimally with iPhones.
- Google kept changing the API, the user interface and the update/installation process. They even changed the name.
- Smartwatches were frustratingly power inefficient in ways we could not alter.
- Manufacturers frequently discontinued the smartwatch models we used, including discontinuing software updates, which eventually bricked them.
- The smartwatches weren't standardized enough that we could write software once and it would run well on all varieties.
The problem is that no wearable platform good enough was already owned by enough of the population. As counterintuitive as our approach might seem to a software developer, it made more sense for us to manufacture our own hardware.
We kept the software. It's somewhere on the cloud in a VCS. Not sure about the data. This was an embedded system. The hardest, most frustrating part to maintain was hardware integration. The prototype was a specific wearable device attached to a specific laptop. I erased that laptop and gave it away to a relative.
In a perfect world, I'd redo this project as a contractor for an established wearables company. I'd do the machine learning, they'd do the hardware and we'd outsource the data annotation to Mechanical Turk. (Data collection is easy. The data bottleneck is annotation.) But that takes industry connections I don't have.
It didn't track calories. It tracked bites. The problem we were attacking was eating awareness and frequency. It might've been possible to track food types too but we didn't get that far.
(Was it a "beep to remind you not to snack" device, rather than a calorie tracker?)
Yes.
Even a tool like this was useful enough that people repeatedly and independently, on their own initiative, asked us to build it for them. They were all from the same demographic: women, often stay-at-home, between the ages of 30 and 50. Their problem, as they saw it, wasn't eating calorie-dense food. It was frequent unconscious snacking.
I appreciate your supportive encouragement. This story took place over a year ago. I have had plenty of time to wrestle with the competing values. This wasn't the first time I chose to abandon a project which potentially could have helped people at scale. I have limited resources. I have to make hard decisions. I like making hard decisions because the act of facing hard decisions implies I'm living life to the fullest.
I don't think I need Karpman's Drama Triangle right not but I do see the connection. It definitely would have helped me if I had read it ten years ago, but that is an unrelated story I do not expect to ever publish.
Those are perfectly reasonable questions to ask but I'd rather not go into the details. Starting a startup requires you to bet hard on complex chaotic uncertain situations. I don't want to paint anyone in a bad light unfairly.
When I look back at this interview and read Sam Altman's blog the common themes which keep appearing are speed, scale and leverage. In our situation, he wasn't just looking for exponential growth. He was looking for exponential growth that was fast even by the standards of exponential growth. The general impression I got (these are my words, not his) was "can you add another zero in a year" followed by "why can't you do it in three months".
That makes a lot of sense. I forgot about the whole "emigration from Britain" thing.
I rarely write about myself but when I do the posts tend to be better received than usual.
Thank you for the link. I'm curious what the table would look like if we examined the top 10 or 20 cities instead of just those tied for the top position.
If the Qing dynasty had to fight the armies of the Napoleonic era in 1789, it's clear to me that they would have been massacred, and I don't think the reason would have much to do with technology.
Who does "they" refer to in this sentence? It could mean two very different things.
Likewise. Your posts were among those that originally impressed me enough go begin writing on this site.
Why was the Western Roman Empire, which fell in 476, instrumental in helping Europe conquer the world in the late 1400s?
Welcome! You don't have to be a "rationalist" to write here.
In my personal experience, writing on Less Wrong has been a boost to my professional network. It has had a tremendously positive impact on my writing quality. The downsides are so low they are unmeasurable. However, I am not a professional academic. Your experience may differ.
When I say "downsides" I'm referring to professional downsides with concrete real-world consequences. It took me a while to develop thick skin but if you're a blogger and a researcher already then nitpicky criticism in the comments may be a non-issue for you.
I think writing under a pseudonym is a great idea. I do it myself, even though my real-world identity is not truly secret. I think using a pseudonym helps me maximize upside while minimizing downside. Using your real given name but no surname puts you in good company; Scott Alexander does the same.
It's a lovely word. Exponential. Exponential exponential. Exponential exponential exponential exponential.
This would be very interesting, if true. Imperial powers (including the USA and Japan) definitely did attempt to industrialize their allies and subject populations. Japan industrialized Manuchuria. The USA helped industrialize South Korea and Taiwan.
My personal theory is that the relative (in absolute terms) prosperity of Japan and USA is what gave them the excess capacity necessary to industrialize a foreign land.
His wider thesis is that worker productivity was higher in the UK due to cultural and genetic changes which had spread 'middle-class values' down through the population and led (in part) to the Industrial Revolution.
I have not read Gregory Clark. What kind of "genetic changes" and "middle-class values" does Gregory Clark write about?
[..] too stupid to invent the key technological advances used against them, and doomed to die because they failed to build cities, domesticate animals and thereby acquire infectious organisms.
Isn't the whole point that stupidity has nothing to do with (e.g.) not domesticating an animal you cannot find in your region?
Yes. This is what Jared Diamond was arguing against. He is very clear and explicit about it.
This is simply false…. In the early Roman imperial period, for example, the biggest cities in the world were Rome, Alexandria, Seleucia and Luoyang…. The same pattern holds later in history also; for example, at the height of the Abbasid Caliphate Baghdad was the biggest city in the world, at the height of the Ottoman Empire Constantinople was the biggest city in the world and so on.
I'd be happy to be corrected if I'm wrong. Do you have more precise numbers?
There's no evidence for the claim that technology regressed after the fall of Western Rome. There were not as many big cities in Europe as there were in the time of Western Rome, but where do you get the impression that technology went backwards in this period?
Roman concrete fell out of use after the fall of the Western Roman Empire. It is my impression that not many aqueducts were built either.
My reference point for technological regression after the fall of the Western Roman Empire comes from science rather than technology. My understanding of the Renaissance (from reading Destiny Disrupted) is that much of European philosophy (including science) only survived because it was preserved by the Arabic-speaking world.
In most cases the reason European armies were able to defeat local armies was not that the locals lacked the equipment the Europeans had; indeed they could buy that equipment from Europeans on the open market!
I agree. This is why Europeans choosing the terms of engagement was so important. They won when the Mughal and Qing empires were at their weakest.
IMU-based gesture detector.
I think you need an explanation for why Europe was first able to do these things while China/Arab states were not.
I think I did address this point. While not "halfway around the world", China did "devote enough resources to fight and win wars half way around the world" first. The Portuguese exploratory voyages started at the end of the 15th century. The much bigger Ming treasure voyages started in the beginning of the 15th century.
To clarify, I agree with Jared Diamond's overall thesis that the interconnected trade networks linking giant cities on the Eurasian biome (including North Africa) produced network effects that gave Eurasia an unassailable advantage over America, Australia and (probably) Subsaharan Africa. In this context, I think of "guns" and "steel" as catchy concrete shorthand for the the more verbose and abstract "technology and heavy industry". I think that Eurasian (including north African) dominance over America and Australia (Subsaharan Africa is more nuanced) was so overdetermined by the 15th century that it doesn't matter to my core thesis whether Jared Diamond was right or wrong about all of his particulars.
Do you disagree with my core takeaway from Jared Diamond that Eurasian (including north African) dominance over America and Australia (and, to a lesser extent, Subsaharan Africa) was overdetermined by the 15th century due to Old World network effects related to technology, disease and industrial capacity stemming from large interconnected population centers?
Other explanations include things like the superiority of the Aryan race and the divine right of kings. There are too many bad explanations to refute. In my personal experience, it's more productive to only engage with explanations I think actually make sense.
You make good point about how "European states [including Britain] without empires also industrialised rapidly". I think this has to do with their proximity to the centers of power of European empires.
I do agree that the European powers still would have set up trading posts in the Indian Ocean if it wasn't for America. I think the discovery of America was like pouring gasoline on a fire that's just getting started. Was it necessary? Maybe not. But technology is a race and I think the American colonies helped European power a lot.
You list four popular explanations (not including mine). Which ones do you think are true and why?
Many competing states with a natural geography full of barriers stopping any single empire from forming and dominating = more competition/experimentation = more progress
I think this theory explains the rapid advancement of European weapons technology and the consolidation of European nation-states. It also explains the advancement of weapons technology in China prior to the consolidation of Ming power.
Property rights and a strong trader/merchant class with a large degree of influence on government vs religious+millitary rule in the arab world. (Note this doesn't apply to all of europe, more to the UK and netherlands. Doesn't explain the success of other European nations) [sic]
It's an interesting theory. This certainly helps explain the lack of commercial development in Japan. (Though commercial and technological development still happened (albeit slowly) in Japan despite the Tokugawa Shogunate's best efforts.) I am skeptical there has ever been a strong trader/merchant class without a large degree of influence on government because wealthy merchants can often buy their way into the lower aristocracy.
The European invention of limited liability companies is especially interesting, not least because of how it sets Europe apart from the Muslim world. But Muslims have invented many imaginative workarounds to escape their religious restrictions. I don't think they'd suddenly draw the line at corporate financial structures.
Unique geographic features such as minimal natural disasters, large amounts of arable land, good climate, lots of large animals and good crops => higher pop density => more innovation and growth
I think this theory is backwards. Europe had fewer natural disasters but its population density is way lower even today. Europe's urbanization rate was below China's until 1800. This is an explanation of why China had better innovation and economic growth for most of history.
European christianity [sic] being in a better state, somewhat de to the reformation, and that having ripple effects throughout society in terms of norms etc...
I am very skeptical of this theory. Christianity and Islam are (compared to Hinduism, Buddhism, Daoism and Confucianism) almost indistinguishable from each other. A schism within Christianity is hair that has been split twice.