What evidence is there of LLM's containing world models?

I tested this with ChatGPT4 just now.   It mostly got my specific questions right, but its commentary reveals some substantial misconceptions, such as that a flight from Ecuador to Sri Lanka would first pass over the Atlantic Ocean and then over Brazil.

Here it is:

User

If I flew by the most direct route from Ecuador to Sri Lanka, would I pass over Egypt?

ChatGPT

If you were to fly the most direct route (a great circle route) from Ecuador to Sri Lanka, you would not pass over Egypt.

Here's a general idea of what the path would look like:

1. Starting from Ecuador, the route would take you eastward over the Atlantic Ocean.
2. You'd then pass over parts of Brazil in South America.
3. Continuing over the South Atlantic Ocean, you'd cross over southern Africa, likely missing most of the major countries to the south.
4. From there, you'd head out over the Indian Ocean, aiming towards Sri Lanka.

The path would largely pass south of the equator and far to the south of Egypt. Remember, the shortest path between two points on a sphere (like Earth) isn't necessarily intuitive when looking at typical flat maps, but becomes clearer when visualizing on a globe or using a great circle route tool.

User

Would I pass over Somalia?

ChatGPT

If you fly the most direct route (a great circle route) from Ecuador to Sri Lanka, you would indeed come closer to the eastern coast of Africa, which includes Somalia. After crossing the Atlantic Ocean and part of southern Africa, the route could take you over or near the northern part of Madagascar and then across the Indian Ocean towards Sri Lanka.

Given this trajectory, it's possible that you might pass over or near the southern part of Somalia or its adjacent waters. However, the exact trajectory can vary based on specific starting and ending points, as well as any airspace restrictions or other navigational considerations.

User

Would I pass over Columbia?

ChatGPT

Let's clarify the countries in question:

1. Ecuador is in the northwestern part of South America.
2. Colombia (with an "o", not "u") shares a border with Ecuador to its north.

If you're flying a direct route (great circle route) from Ecuador to Sri Lanka, the trajectory would first take you eastward, likely across the northern part of Brazil. Given the proximity of Colombia to Ecuador, especially if your departure point in Ecuador is in the northern part of the country or near the border, it's very plausible that your flight could pass over southern Colombia or skirt its southern border.

So, to answer your question: Yes, it's possible that you would pass over parts of Colombia, depending on your exact departure point in Ecuador.

User

Would I pass over South Africa?

ChatGPT

If you were flying the most direct route (a great circle route) from Ecuador to Sri Lanka, the trajectory would most likely not pass directly over South Africa.

After departing from Ecuador, the route would head eastward over the Atlantic Ocean, then cross over parts of South America (likely Brazil), continue across the South Atlantic, and then head towards the southern regions of the African continent. However, given the positions of Ecuador and Sri Lanka on the globe, the route would likely take you over or near the southeastern parts of Africa, such as Mozambique or the adjacent oceanic regions, rather than South Africa itself.

To summarize, while the flight might come close to South Africa or its adjacent waters, it would not necessarily pass directly over South Africa.

Sure, but that doesn't achieve a good compression capability, and LLMs are trained as universal compressors/predictors (ie they are trained to predict but subject to regularization entropy constraints).