Chicanery: No

I think that this is a bad concept.

I think this primarily for anti-Gell-mann-amnesia reasons, i.e. I am very sure that this is a very bad concept as applied to TTRPG design/implementation (a topic on which I am quite knowledgeable), so I conjecture, even before thinking about it directly, that it is also a bad concept as applied to the other things that you are applying it to (in which I would not claim as much expertise). (My weak impression from thinking directly about those other cases agrees with this, but that is not the main thrust of my comment.)

In the rest of this comment, I’ll explain why I think that this is a bad concept as applied to TTRPGs.

In a well-designed TTRPG of comprehensive scope (such as most editions of D&D, and Pathfinder), the optimal amount of “chicanery”, as you define it, is simultaneously “absolutely none whatsoever” and “the maximum possible amount”, in all circumstances. Obviously, this cannot be true unless “chicanery” is a nonsensical, contradictory, or otherwise useless concept.

But why do I claim that this is true?

The ideal way that things like alter self should work is that they should be completely robust to attempts to “rules-lawyer” or “break” them, while also implementing the fun fantasy trope and not being very complicated, while also allowing for player creativity and flexible usage.

The way that these desiderata can be satisfied simultaneously in a single design is for the DM to have a consistent, coherent, and complete world-model, and for all answers to player questions about the world as apprehended by their characters, and all responses to declared player character actions (those being the exactly two ways in which players of a traditional roleplaying game—i.e. one with no or almost no dissociated mechanics—may interact with the game world), being generated by querying that world-model. Under this view, game rules are understood to be components of the definition of the DM’s world-model (and should be written accordingly—a dimension along which D&D and related games has certainly varied, between and even within editions and versions).

If this approach is done correctly, then you can, at all times and in all circumstances, both “do the obvious, sensible thing” (as far as interpreting any ability or other aspect of the game world), and allow players to “explore edge cases” and otherwise optimize as they please. This is very easy, when the answer to all questions about how a character ability (or anything else whatsoever) works is a query to the DM’s world-model. What does this ability actually (in the game world) do and how does it do it? Whatever is the answer to that question, that determines what you can do with it.

(This approach—which has some overlap with what is sometimes called “simulationism”, but is not identical thereto—also creates the maximum amount of player engagement and satisfaction, is the best at allowing newcomers to engage with the game, permits flexible switching between adventure and campaign styles, allows relatively easy reuse of old material even across editions, and has various other desirable qualities besides.)

This approach maps to “maximum chicanery”, because it allows players to push everything as hard as they please, look for edge cases to exploit, optimize, etc.

It also maps to “no chicanery”, because the DM interprets everything in the most conservative way possible: “what does this actually do and how does it actually work, within the game world” is the way that all questions are answered. (Note that this applies when interpreting the rules, as distinct from designing the rules; while a DM may take on the latter role, possibly quite often, the roles must be cleanly separate conceptually.)

Let’s consider the hypothetical form-changing ability example again:

Imagine a magic power in a roleplaying game that says “You may change your features to look like another humanoid. You can change your entire appearance, even your sex, but cannot look like something significantly larger or smaller, or anything that isn’t humanoid.” Imagine it being positioned as a basic disguise power, not too high level.

And suppose we decide that this form of magic changes your external qualities, but not your substance; you’re still made of the same stuff, just rearranged somewhat. We also consider how our world works, and decide that people can’t use magic (that they are casting themselves—as distinct from, say, drinking an unidentified potion) to turn into something that they don’t know about or haven’t seen.

And now let’s see how we might respond to some of the edge cases you list:

Can I change my features to have gold fingernails, clip my fingernails, and sell the gold?

We query our world model, and recall that you’re only changing your appearance, and bits separated from a whole that is under the influence of magic aren’t themselves under the influence of said magic anymore, so revert to their original forms.

Can I change myself to look exactly like the queen?

The question here is “is this a sort of magic that gives you fine control over details”? That’s a designer’s question, not an interpreter’s question. If the answer is “yes”, then—yes, with a suitable Disguise skill check (or equivalent), you can take on the semblance of the queen. Otherwise, nope.

How much larger is “significantly” larger anyway, can I get tall enough for the extra reach to matter in a fight?

Since you’re just rearranging the stuff you’re made of—probably not. (Note that if the system we’re using has size categories, then “significantly larger or smaller” would mean “different size category”, and the ability would probably be written that way.)

Can I change into someone much more attractive and get a charisma boost?

Charisma doesn’t come from appearance, so no.

Can I change into a humanoid creature with claws

Are there humanoid creatures with claws that you know about? If so, then yes, of course. Otherwise not.

and cut myself free of these ropes I’m tied with?

Can claws cut ropes? If so, then yes, of course. (And do the clawed humanoid creatures that you know about have the kinds of claws that can cut ropes? Not all claws are the same, after all…)

Can I change into a magic humanoid creature whose hair is on fire and have my hair be on fire, then light a match off it?

Are there magic humanoid creatures whose hair is on fire that you know about? If so, then yes, of course. Otherwise not.

Can I change into a humanoid creature with wings, then fly?

Are there humanoid creatures with wings that you know about? If so, then… etc.

Hey, I just got stabbed- can I change into me, but without the open wound?

This would be another instance of that “fine control over details” question above.

If so, does that heal me?

Well, it could hardly put the blood you already lost back inside you, but (if sufficiently fine control is allowed) it could stop bleeding. (Is there bleeding in the system we’re imagining?) Does that translate into “healing”? Depends on other implementation details of the system. (“What do hit points represent” is a much larger discussion.)

What we’ve learned from this exercise is that the description of the ability, as given, is insufficient to unambiguously specify a model specific enough that further details may be extrapolated from, but that the amount of additional specificity required for that is not very large. For example, the given description says “to look like another humanoid”, but does that mean “a specific individual humanoid [e.g., ‘Bob, the half-elf accountant’]” or “a kind of humanoid [e.g., ‘hobgoblin’]”? This isn’t a concession to rules-lawyerly pedantry, note; it’s a genuine ambiguity even in natural language, quite apart from any TTRPG application.

And this is applicable broadly. Famously, the D&D rule books (in some/most editions) don’t say that dead people can’t take combat actions. So (says the rules lawyer), can I keep fighting, even though I’m dead? We query our world-model and answer: obviously dead people cannot take combat actions, because they’re dead. What if (says the creative out of the box thinker) I want to murder someone, so I stuff their roast chicken full of rat poison—does this kill them dead, even though normally one does get a saving throw against poison and the effect isn’t “death” even on a failure? Well, the rules technically don’t have any provision for “stacking” doses of poison to increase their saving throw DC, or… but our world-model says: no, don’t be silly, of course that works; that guy is gonna be as dead as that roast chicken.

You mention Microlite, where “rules-lawyering seems rude”. No! Rules-lawyering is no more or less rude in Microlite than in full-blown D&D 3.5! In both cases, we answer all questions—all of them, every single one, without exceptions—by querying the DM’s world-model. Microlite just gives the DM much less guidance for constructing that world-model than D&D 3.5 does (which is the price for having the rules be orders of magnitude faster to read and learn). If the hypothetical “rules lawyer” insists that the rules override the DM’s world-model—wrong. Wrong, in both Microlite and 3.5, equally. The rules constitute (part of) the definition of the DM’s world-model (as well as playing a large role in communicating the world-model to the players).

The DM’s world-model is the ground truth. Answers to all questions flow from it.

This completely eliminates “chicanery” as a coherent concept.