Starting too many projects, finishing none

A potential way to think about this:

I learn a lot from most projects, and I think this is a huge benefit from at least starting a project.  I then have whatever I learned to use in the projects I do end up taking to completion.

Also remember that most projects are likely to be failures in some way (at least I think this is the case, I don't have the data lined up to back this up).  Once you've squeezed much of the learning potential out of them, then it's likely you're not losing much, if anything by dropping the project.

Find some overarching theme to the ideas you are most excited about. Make your ideas into a roadmap towards whatever the ultimate achievement of the overarching theme would be. Then take the smallest possible step along that road. And then the next. 

Forget about "high expected-value projects". Think about what you want to be able to do long-term. Pick projects that allow you to gather the experience and aquire the skills to do this. You motivation shouldn't be "if this project were completed it would be really cool" - it should be "everytime I work on this I gain skills/knowledge/connections/... that will lead to me being able to long-term complete many such projects".

I also have this experience. For me the answer is to work with other people. In those weird personality test things that businesses sometimes do their is a category of person (I think, for some reason, called a "plant") who flits from one exciting idea to another and never finishes things. The silly business teamwork talk thing suggested that a good team has someone like that, then someone who is a bit more negative who can try and shoot ideas down to make sure bad ones don't get through, then finally a person (possibly the second person again) who is dedicated to making sure existing things get finished and pushes for that.

In my own experience, my personal projects flit all over the place. But in team situations I am forced to commit.

I’m glad you posted this — this may be happening to me and now I’ve read about sunken cost faith counterfactually

You mention paper ides, so I will answer assuming this is about research.

In my experience, picking a good research topic up front is extremely important, since actually testing your idea in depth will take a lot of time and there's a significant chance that once you do that you'll have to discard it anyway. So I think a period of flipping through project ideas without long-term commitment to any of them is pretty helpful for feeling out whether a given idea is worth the time and risk. Just make sure that you really seriously consider each idea you're trying out rather than just dropping it for something else, and write down what you're thinking in case you want to revisit the idea later.

During this trial period for an idea, I'd suggest biasing towards feeling out what the project might look like in the longer term rather than diving directly into implementation, though doing some initial prototyping can help you learn quickly here. Think about potential impact, feasibility given your time and resources, any unknowns you think could sink the project (investigate these ASAP), anyone you might want to collaborate with, and how it might fit into a broader research landscape. Talk the idea over with other people. Try to learn quickly here, and move on freely. Understand that if you commit to a project, there will likely be months of work before any payoff.

I have also heard the advice to keep multiple projects going simultaneously, although I haven't tried this myself. I would caution against using this as an excuse to backburner a bunch of things rather than focusing down and killing what isn't working, though.

One possible explanation+solution from the "Hammertime" author: https://www.lesswrong.com/posts/o8EWXypYGwJ6JYpBT/hammertime-day-8-sunk-cost-faith

I try to budget some time/energy for new ideas, new projects, etc. So that I can satisfy my unsatiable hunger for "new" without feeling guilty about it, but at the same time can make progress on some longer term project with the majority of my time/energy. 

Redacted : removing identifying information