Avoid the abbreviation "FLOPs" – use "FLOP" or "FLOP/s" instead

"Floating point operations per second" is usually abbreviated FLOPS. I think using FLOPs as a synonym for FLOPS seems wrong in a particularly confusing way. It really suggests that FLOP is the acronym and the "s" is for a plural. Compare to someone saying SMs instead of SMS.

That said, I agree that using FLOPs as the plural of "floating point operation" is confusing since it is pretty similar to FLOPS. It's also not something I've seen much outside of the futurist crowd (though I am guilty of using it fairly often). Another way in which the ML and futurist usage is nonstandard is that we often use FLOPS sloppily to talk about very low precision or even integer arithmetic.

I haven't personally seen this cause very much confusion, in my experience it maybe increases the amount of "compute stock vs flow" confusion by like 25-50% from a small-ish baseline? I suspect it's small enough that accessibility and using standard notation is significantly more important. (Though again, my sense is that FLOPs for "floating point operations" is not really standard and not particularly accessible, so it seems more plausible futurists should stop doing that.)

Not sure what's realistic to do here. I'm a bit inclined to use "operations" or "ops" to both avoid this ambiguity and be more inclusive of integer arithmetic, but I kind of wouldn't expect that to stick and maybe it's bad in some other way. Using FLOP as a plural noun is pretty unsatisfying and I suspect it is unlikely to stick for fundamental linguistic reasons. Switching to using FLOP/s instead of FLOPS may be realistic, though I think flop/s is a bit less weird-looking to me.

ETA: though I think openphil has been using FLOP/s and FLOP as you suggest in their writing, so maybe I'm wrong about whether it's realistic. I think the draw to add an s to plurals is strongest in spoken language, though unfortunately that's also where the ambiguity is worst.

Can you point to example usages that are causing confusion?

Agree that this discussion is surprisingly often confusing and people use the terms interchangeably. Unfortunately, readers often referred to our training compute measurement as a measure of performance, rather than a quantity of executed operations. However, I don't think that this is necessarily due to the abbreviations but also due to the lack of understanding of what one measures. Next to making the distinction more clear with the terms, one should probably also explain it more and use terms such as quantity and performance.

For my research, I've been trying to be consistent with FLOPs (smaller case s) referring to the quantity. While FLOPS or FLOP/s refer to the performance: operations per second. (FWIW, during my time in computer engineering, it has been the norm to use FLOPs for quantity and FLOPS for performance.)

The term Petaflop/s-days also helps - outlining how long the performance (Petaflop/s) runs for how many days, therefore measuring a quantity of operations.

Note it gets even more complicated once we take the number representation (floating point 32bit, 16bit, or even bfloat16) into consideration. Therefore, I'm also in favor of maybe switching at one point towards OPs and OP/s and also document the used number representation for actual technical documentation (such as reporting the compute of ML models).

This seems like a good idea, although I feel that the number of people who use FLOPS to mean FLOP per second are still going to grow faster than those who switch to using FLOP/s, as most introductory material from a simple google search still refer to FLOPS as FLOP per second. Perhaps an edit to the wiki page could expedite the switch? 

Also, will FLOP/s still be pronounced 'flops' or will it now be 'flop per second'?

i can’t think of another (pseudo-)acronym which gets used as an all-caps unit, off the top of my head. i may toy around with “Flop” as a unit, like GFlop for a billion operations, GFlop/s for a billion operations per second.