Comment by benmiller00 on [Link Post] Bytes Are All You Need: Transformers Operating Directly On File Bytes · 2024-03-11T12:04:04.642Z · LW · GW

The ByteFormer model you're discussing sounds pretty groundbreaking! It's fascinating how it can classify images directly from TIFF file bytes with such high accuracy (77.33% on ImageNet), beating traditional methods that work on RGB images. Even more impressive is its ability to handle WAV files from the Speech Commands v2 dataset with minimal effort, scoring a 95.42% classification accuracy.

The part about privacy-preserving inference is super intriguing. Operating on obfuscated inputs without losing accuracy could be a game-changer for data privacy. The idea of a privacy-preserving camera that ByteFormer can work with, even when 90% of pixel channels are masked, yet still pulling off a 71.35% accuracy, is pretty cool.

Regarding the PNG to PDF converter link (https://oneconvert.com/pdf-converter/png-to-pdf, it seems a bit out of context here unless you're thinking about applying ByteFormer to document formats too, which could open up some interesting possibilities in document classification and processing.

All in all, ByteFormer seems like it's paving the way for more efficient and versatile deep learning models, especially in terms of handling different data types and protecting privacy.