When does something stop being a “self-consistent idea” and become scientific fact?

Of course the limited amount of knowledge available to the primitive tribe doesn't rule out the existence of George, but neither does it do much to justify the theory of George. What they know is that the ground shook, but they have no reasonable explanation of why.

There are, for them, many possible explanations they could dream up to explain the shaking. Preferring any one above the others without a reason to do so is a mistake.

At their postulated level of sophistication, I don't think they can do much better than "The Earth shook. It does that sometimes." Adding the bit about George and so forth is just unnecessarily multiplying entities, as Ockham might say.

Your four criteria leave an infinite set of explanations for any phenomenon. Including, yes, George the Giant. That's why we have the idea of Occam's razor - or, more formally, Solomonoff Induction. Though I suppose, depending on the data available to the tribe, the idea of giant humans might not be dramatically more complicated than plate tectonics. It isn't like they postulated a god of earthquakes or some nonsense like that. At minimum, however, they are privileging the George the Giant hypotheses over the other equally-complicated plausible explanations. The real truth is that they don't have enough data to come up with the real answers. They need to start recording data and studying the natural world. They can probably figure it out in a few hundred years if they really put their backs into it.

I would stay away from the accept/reject terminology and simply say they should assign low probability to this theory due to Occam's razor, and increase the probability if it survives experimental tests.

For comparison, it happens on occasion IRL: neutrino. Pauli was said to have some qualms about proposing an undetectable particle.

The trick is to stop thinking in terms like

something we can believe

and

accept it into their belief system

Ideal rational entities just assign probabilities to each possibility (using Occam's razor if nothing better suggests itself), and then update using Bayes theorem when they see evidence. They can then make action in light of these probabilities (trying to maximise some utility function). The points on your list all follow from trying to obey the laws of probability and utility maximisation. In the case of the earthquake there are a variety of explanations, some of them are fairly likely ("nearby avalanche", "large creatures (e.g. George) moving about", etc. ), some are fairly unlikely ("plate tectonics", "attack by a technologically advanced tribe") and some are very unlikely ("God did it", "random movement due to thermal vibrations"). As they gain more evidence their beliefs will change. But they never have to accept one belief or another, they can quite happily maintain their probability distribution and be honest about their ignorance.

Also note that their probability distribution can be useful even if it is spread across many possibilities (i.e. if they are very ignorant). For example, if all of the most likely hypotheses predict recurrent but infrequent earthquakes, then they can be confident in this prediction even though they don't know which of those hypotheses is correct.

George the Giant or invisible cosmic springs that are too small to ever measure? Also a bunch of extra spatial dimensions that information can travel through without us being to see it. I see what you did there.

Ultimately I'd say there is nothing wrong with the primitives thinking, as long as they are willing to upgrade their model as better evidence becomes available. When the primitives finally send someone to check the other side of the mountain and see no giant, they need to eventually assume that the cause must be somewhere else they cannot see, like under the ground.

Also, if they think of a simpler model RIGHT NOW that meets all the above restrictions, they should "upgrade" to that simpler model because it saves computational time to work with it.

When the primitives develop tools to actually see the ground moving, and find out it does this all the time, they have to upgrade their model further to realize that somehow earthquakes are a property of the ground itself.

And so on up to present, and possibly the future when we can model the entire earth as a series of potential energy reservoirs and predict precisely when stress levels are building to the point of an energy transfer occurring.

I agree that Occam's razor is clearly the answer, clearly what was missing to support the George Hypothesis. I didn't mention it in the original post because I can't really quantify what is meant by that, and it kind of felt like a thought-stopper. 2 important questions:

1) When/how/under what conditions would you consider Occam's razor satisfied in this example? (Feel free to add onto the fictional scenario). What if giants really did exist (though were few in number), and the tribe had seen one as recently as 20 years ago? [In this example, the earthquake is still caused by plate tectonics, not giants, but the tribe has never heard of such things.]

2) Is it ever acceptable to take on a belief like this, even if you're not sure it satisfies Occam's razor? Remember - the belief is USEFUL to these people - over the next 200 years it allows them to make apparently accurate predictions about the future (that the rumblings will continue sporadically, as opposed to being a 1-time thing)

Real-world analogies:

-Shminux mentioned neutrinos - GREAT example (link to his comment: http://tinyurl.com/97tkabf); as we CAN detect those nowadays, but at the time most folk couldn't think of a good test to confirm/deny it.

-"Gravitons" spring to my mind - I don't remember much quantum physics from undergrad, so I'm not on a position to judge, but in my ignorant opinion they seem like a very similar scenario - fit with the model of quantum physics, not much evidence against the idea, potentially useful, but not a lot of actual evidence (empirical or otherwise). (Would the Higgs Boson be a better metaphor?)

I like this as a parable. I've been talking to several people trying to explain cosmology and Occam's razor and why the default position should be "I don't know" instead of "I don't know therefore X" but they just don't seem to get it. Instead of trying to start the conversation with life, the universe, and everything, I should probably start with an example like the George the Giant story above. It should be relatively easy for them to see why George the Giant was a bad belief, even though it satisfied the four criteria (i.e., because it fail's Occam's razor and there's no reason to elevate it above an infinite number of similarly complex answers)

We see something in the world that appears mysterious to us, and we come up with an idea (“idea X”) that explains it.

We can come up with a scientific theory that is not based on observation. Most theories aren't based on observation. Observation comes after. 'I bet I can jump that fence' comes before observing it.

be internally consistent with itself, as well as with all its implications. i.e. if X, then Y, and if Y, then Z. If we know Z to be obviously false, then we know X must be false.

Good!

be externally consistent with reality as we know it. We can’t find something in reality that makes X clearly unture.

Untrue not unture, otherwise great! Falsifiability is the component that makes a scientific theory different than any other theory.

Explain things we have already observed (that’s why we’ve come up with idea X in the first place)

Good! A valuable new theory will incorporate prior theories, including their errors and omissions, then add to them and address the errors or omissions. Again, all this can happen without observation (a lack of self-consistency, for example)

preferably, “make beliefs pay rent” – we should be able to use X to make predictions of the future, otherwise it doesn’t hold much value.

Good! Perhaps one could add the value of paying a good rent instead of a morally neutral rent (ie better homes over better bombs), but perhaps not.

An ommision in your work: scientific theories are all (all) provisionally true [edit: if they are held to be true at all]. Aside from limits of money / time / resources, it is always (always) acceptable to challenge a scientific theory. Anyone can, at any time, with any background. Results will be mixed and often support conventional wisdom. But not always.

"Conjectures and Refutations" by Karl Popper has what you're looking for.