Luckily, it seems we also have some specific signals to tell us rich vs poor, specifically conditioned on taste. This might alleviate the effect significantly, though perhaps not fully.

The problem starts to look ill-posed if you look at all the factors, though. Of course better taste has to correlate some with actual wealth, as if it provides any value at all it will give those people better social connections, etc. This whole cluster of value-providing traits is in fact correlated—and specifically because they're correlated does this central dynamic exist, that wealth and taste (and any other value-providing trait) can both be traded in for a costly signal. So to postulate a world in which they don't correlate would be postulating a world in which this dynamic doesn't occur.