What keeps the earth from collapsing is not electrostatic energy (well, I guess you could probably describe it so it sounds that way, but I don't think that's the best way to describe it) and no, it isn't used up.
At a microscopic level, things do move around in various ways even when they're part of objects we think of as being in equilibrium. Heat is the name we give to this sort of jiggling about, and it's a form of energy. But in equilibrium the net force acting on those atoms is zero, so the jiggling isn't moving things against an opposing force.
("But wait, say one of those atoms jiggles just a tiny little bit. Then it's no longer in its equilibrium position, which means that any further jiggling will be against an opposing force." Ah, but further jiggling may move it the other way, in which case the atom is gaining energy instead of losing it. On average it stays in the same place and the net energy change adds up to zero.)
In the very long term the jiggling will reduce, not because energy is being used up holding things apart but because heat tends to spread itself out uniformly (the jiggling spreads to other nearby things) and the average temperature of the universe (temperature is a measure of heat per available way-of-moving, which is kinda like heat per molecule) is rather small -- so in the long run everything gets pretty cold. In the short term, the dumbbell is at about the same temperature as the other stuff around it, and the flow of heat out of it is matched by the flow of heat into it.
But the key thing here is that exerting a force, as such, does not consume energy. The earth is not radiating away energy as it exerts a gravitational pull on everything else in the universe. An electron is not radiating away energy as it exerts an electrostatic attraction or repulsion on every other charged object in the universe. This may be counterintuitive if you think of energy as that stuff that everything needs in order to do anything, but the moral is that that isn't quite the right way to think of energy.