Leave rocks in soil or try to get them out?

I live in the Rocky Mountains and every time I dig my shovel strikes rocks, either big or small. Sometimes they are big and flat which makes for nice stepping stones, but the rocks are bountiful and of all sizes all the way down to pea gravel. I’ve been obsessed with removing them from the soil before planting anything.  I’m wondering if this obsession is unwarranted.

It’s easier to get the big rocks out, simply because there are so many small rocks I need to sift them out which is very time consuming.

I know rocks on top of the soil could have advantages. But are rocks an issue inside the soil?  Do they take up room which is otherwise healthy soil?  Or should I not even bother with sifting our rocks or removing the larger ones?

If they are smaller than a quarter or so, I leave them.

The main reason I understand people want to remove rocks from their fields is to make life easier on soil working equipment. So I figure you only need to remove the rocks that make it hard for you to work your soil. Rocks add thermal mass to your soil, they can create microclimates conducive to soil.microbes, they are a slow release source of minerals. I wouldn't worry about sifting anything out, the larger and easier to remove ones could most likely be used to greater advantage elsewhere

Depending on your geology it might be a case of rocks all the way down!

In rocky areas of the UK the traditional way to improve rocky land for planting was to use the stones to build dry stacked walls. Simultaneously building a fence and making working the land easier. You have already identified that the bigger rocks are the main problem, and you are probably on a hiding to nothing extracting the smaller ones. The exception might be if you were trying to improve smaller areas - like raised beds - for a garden. On a large scale such a task would be futile.

Hey Seth!
In my feeble attempts to be like Sepp, I begin to see all sorts of uses for the larger stones out of the ground placed tactically to do different jobs (windbreak, heatsink, small critter habitat, etc), and the smaller ones in the soil for their ostensible mineral content.

I'm sure a big part of the decision is your effort:payoff ratio. Do what gets you your biggest bang for your buck.

Maybe share with us what you decide to do, or some of your stonework projects!