Looking for books about software engineering as a field

I have an intuition here that learning to code might be almost necessary for what you want. It's only an intuition, and it's not very strong. You may feel like your current understanding is higher than this intuition would predict, and I wouldn't contradict you. But it seemed worth sharing.

My feeling is that trying to understand these things without knowing how to code, would be like trying to understand the classification of finite simple groups without having sat down to play with some examples of groups. One could probably get an intellectual understanding of a group without playing with them, but playing with them will give an intuitive understanding that will be super helpful for understanding a "simple group" on an intellectual level, let alone an intuitive one. And so on.

(The rough hierarchy here, as I see it: a "group" is a collection of objects closed under a binary operation satisfying certain properties (examples include integers with addition, real numbers with multiplication, states of a rubix cube with side-twists). A "finite group" is probably easy enough, a "simple group" is one with no "normal subgroups" except the ones considered trivial. To classify these groups requires us to understand "isomorphisms" between groups: the goal is to take a relatively small collection of groups, and say "any finite simple group will be isomorphic to some group in this collection".)

And so my worry is that the foundation you need will need to be intuitive, and not just intellectual; and the way to get an intuitive understanding would be to work with code. (Which is also more than just writing it. Like half the things you name are related to the problem of "running code on a computer different than the one it was written on".) Not necessarily to a high skill level, but to some extent.

Unfortunately, if this is true, it's not likely an easy road. I think I'd been programming for some time before I felt like I understood what an API was. (Not just programming, but actually using APIs.)

For example, I've had three people try to explain exactly what an API is to me, for more than two hours total, but I just can't internalize it.

Perhaps because they rehearsed their understanding at you rather than being more Socratic?

How would you describe what an API is (given your current level of understanding)?

If the books won't satisfy you, you can still ask individual questions here. But, as philh said [LW(p) · GW(p)], there is a chance that fully understanding something requires one to actually use it. Otherwise, your understanding will only be powered by analogies, and it will stop at the moment when there is no convenient analogy for something complex (that is, complex for people who never used it, but kinda intuitive for people who use it regularly), or you may misuse the analogy beyond its purpose. Also, you will be unable to distinguish between correct answers and wrong answers. -- That said, I am curious how far you actually can get like this.

For example, I've had three people try to explain exactly what an API is to me, for more than two hours total, but I just can't internalize it.

API is a list of functionality you are supposed to use (because the authors guarantee it will keep working tomorrow), as opposed to functionality that is either inaccessible to you (therefore you can't use it directly) or is technically accessible but you shouldn't touch it anyway, because the authors make no guarantee they won't change it tomorrow.

More generally, this idea of distinguishing between what is meant for use by others, and what are the internal details the others should not touch, is called "encapsulation".

Encapsulation can happen on multiple levels. You have small units of code, let's call them classes, which provide some functionality to the outside world, and keep some private details to themselves. Then you can compose a larger unit of code, let's call it a module, out of hundred such classes. Now there is a functionality that this module as a whole exposes to the outside world, and some details it wants to keep private. Then you compose the entire program or library out of dozen modules, and it provides some public functionality. API usually refers to the public functionality on the level of program or library.

Analogy time:

Imagine that I am a robot that offers some useful service. For example, I can remember numbers. The officially recommended way to use me is to come to my desk and say "Remember the number X" (some specific number, such as 42) and I will remember it; later you can come to my desk and say "Tell me the number I told you the last time" and I would tell you the number (42, in this case). The list of commands you should officially use with me is my API.

You can either use my services as a person, or you can send your own robots to interact with me. My behavior is the same in either case.

Now you are a curious person, and you notice that when you tell me a number, I will write it down on a piece of paper. When you ask me later, I will read the number from the paper. This inspires you to make an improvement to your process. You tell your robots that instead of asking me about the number, they should simply look at my desk and read the number on the paper. This is 40% faster, and that makes you happy!

Five weeks later, your factory stops producing stuff. It takes you a few hours to find out why. The robots that occassionally come to use my service, keep staying frozen at my desk and never return. That's because there is no paper on my desk anymore.

You complain to my owner and threaten to sue them. But my owner shows you the original contract, which specifies that you (or your robots) are supposed to ask me about the number; and there is nothing there about a paper on the desk. When you ask me, I give you the right answer. It's because this morning I was installed a new memory, so that I don't need to write things down anymore. (Which by the way makes me now 80% faster than before.) All other users of my services are happy about this change. Only you are mad, because now you have to reprogram all your robots that interact with me, otherwise your factory remains unproductive. It takes you three days to reprogram your robots, which means a great financial loss for you.

End of analogy.

The lesson is that when the user limits themselves to stuff they are supposed to use, it allows the service provider to make improvements, without breaking things. The only way to allow future improvements is to make some parts of the operation forbidden to use for the customers; otherwise you could not change them, and you can hardly improve things if you are not allowed to change anything.

If your background is not software related, what is it. It might also help if you shared a bit more about just what writing you will be doing.

I tend to think philh is correct, you probably need to have some understanding of coding to be able to understand the higher level aspects and how things might relate. Then again, you might be able to borrow concepts from other areas you do know about that can serve as metaphor for what you're trying to understand in software engineering related subjects you are writing about.

Not sure if you've already looked at things like:

https://en.wikipedia.org/wiki/Software_engineering

Also, you might take a look at the introduction to some CE textbooks, probably free looking at Amazon, and get something of an industry overview (but that seems like such a broad level it might be meaningless for you goal).

For example, I've had three people try to explain exactly what an API is to me, for more than two hours total, but I just can't internalize it.

Many websites 'serve' some 'purpose'. You access that service through a web browser (while you have internet access). But if you wanted a program (or an app) to navigate a website, that might be difficult. APIs are 'Application (App) Programming Interfaces'. If you don't want to teach a program how to navigate a website*, you can instead understand how the website's API (if it has one) works, and then write a program that 'talks to' the website in that language, which is more minimalist. The way you interact with websites, they are very expressive - beautiful, but sometimes slow to load. When a program talks to the website via an API ('instead of' a web browser), it can get a response faster, and in a form it finds easier to read. (Unless your internet is out, or the website is down.) Part of how this works is that a website gives you everything as an image, whereas an API only answers specific questions when your computer asks them.

If you've ever seen a website with a youtube video stuck in the middle of it, then the website was probably made using the youtube API, because that's easier to do than the alternatives, like creating a video website yourself.

*This may seem do-able, but if you want to compare prices of products between websites, do

as a cohesive whole

I hope such a guide exists, but those things you mentioned may more easily fit together in smaller categories. 'The whole bucketload of terms you've learned' aren't all things that one person knows and handles by themself in every case.