LessWrong 2.0 Reader
View: New · Old · Top← previous page (newer posts) · next page (older posts) →
← previous page (newer posts) · next page (older posts) →
Edit log:
2024-04-30 19:31 CST: Footnote formatting fix and minor grammar fix.
alok-singh on Slick hyperfinite Ramsey theory proofiLate reply, but the slicker bit is going in more fully. The appeal of the NSA approach here is axiomatizing it which helps people understand because people already know what numbers are, so 'inf big' is much less of a stretch than going the usual crazy inference depth math has.
wilkox on Open Thread Spring 2024Katelyn Jetelina has been providing some useful information on this. Her conclusion at this point seems to be 'more data needed'.
leogao on Improving Dictionary Learning with Gated Sparse AutoencodersAnother question: any particular reason to expect ablate-to-zero to be the most relevant baseline? In my experiments, I find ablate to zero to completely destroy the loss. So it's unclear whether 90% recovered on this metric actually means that much - GPT-2 probably recovers 90% of the loss of GPT-4 under this metric, but obviously GPT-2 only explains a tiny fraction of GPT-4's capabilities. I feel like a more natural measure may be for example the equivalent compute efficiency hit.
leogao on Improving Dictionary Learning with Gated Sparse AutoencodersGot it - do you think with a bit more tuning the feature death at larger scale could be eliminated, or would it be tough to manage with the reinitialization approach?
leogao on Improving Dictionary Learning with Gated Sparse AutoencodersMakes sense that the shift would be helpful
leogao on Improving Dictionary Learning with Gated Sparse AutoencodersThanks, that makes sense
redman on Thoughts on seed oilUnfortunately, I haven't found a solution that scales, and I don't think there is one.
I suspect that a clean environment is incompatible with most technological infrastructure. Microplastics, oilfield brines, combustion products, industrial/agricultural/mining waste, etc all accumulate in the environment and concentrate on the way up the food chain. Even a strip mall generates a ton of pollution in the nearby water table.
I've given up on 'pure' and just try to have a clear understanding of how I'm poisoning myself. The most depressing thing about this is that I've been to absolutely beautiful farms, with happy animals...and in my due diligence discovered that the reason it was affordable to homestead there is because the textile plant closed in the 70s, so all the jobs left...but the PFOAs stuck around.
So... I try to know what's going into my body, avoid poison where possible, and do my best to get whatever garbage is accumulating out.
That being said, I think the stuff that has done the most damage to my body are medical products. Read those labels carefully!
measure on faul_sname's ShortformIn this context, 'robustly' means that even with small changes to the system (such as moving the agent or the goal to a different location in a maze) the agent still achieves the goal. If you think of the system state as a location in a phase space, this could look like a large "basin of attraction" of initial states that all converge to the goal state.
jenniferrm on Ironing Out the SquigglesI actually kind of expect this.
Basically, I think that we should expect a lot of SGD results to result in weights that do serial processing on inputs, refining and reshaping the content into twisted and rotated and stretched high dimensional spaces SUCH THAT those spaces enable simply cutoff based reasoning to "kinda really just work".
Like the prototypical business plan needs to explain "enough" how something is made (cheaply) and then explain "enough" how it will be sold (for more money) over time with improvements in the process (according to some growth rate?) with leftover money going back to investors (with corporate governance hewing to known-robust patterns for enabling the excess to be redirected to early investors rather than to managers who did a corruption coup, or a union of workers that isn't interested in sharing with investors and would plausibly decide play the dictator game at the end in an unfair way, or whatever). So if the "governance", "growth rate", "cost", and "sales" dimensions go into certain regions of the parameter space, each one could strongly contribute to a "don't invest" signal, but if they are all in the green zone then you invest... and that's that?
If, after reading this, you still disagree, I wonder if it is more because (1) you don't think that SGD can find space stretching algorithms with that much semantic flexibility or because (2) you don't think any list of less than 20 concepts like this could be found whose thresholds could properly act as gates on an algorithm for making prudent startup investment decisions... or is it something totally else you don't buy (and if so, what)?