Ah! Were you the one who decorated the Rose Garden Inn? I'm really curious how you made the lighting that looks like the sun coming through a cracked door / coming through the cracks between bricks.
Picture below -- I took a ton of pictures when I was there to steal your interior decoration ideas.
Aside which the original author may be interested in -- there has been some work done to reduce the scaling of the context window below O(n^2) -- e.g. https://arxiv.org/pdf/1904.10509v1.pdf. I also think of OpenAI's jukebox which uses a hierarchical strategy in addition to factorized self-attention for generating tokens to effectively increase the context window (https://openai.com/blog/jukebox/)
