I know people have been making stone basements (especially for storing potatoes, etc) for hundreds of years, but what did they hold it together with? That didn't leak water when it rained hard?
Great question, and a wonderful way to do it if you are going to have a basement at all. I typically do not recommend basements under architecture unless you are going to use that space as root cellar, and/or have a walk out basement.
The first thing I point out to folks considering this choice, is understand that it is not going to be easy, or the least expensive (most likely) unless you do all the labor yourself. Even then stone in the hands of an amiture, is going to be very challenging (but great fun!!!) I would also point out (for you peace of mind and a way to speak to folks that try and tell you this isn't a good idea) that we have been building this way for thousands of years. So lets see if I can't break down some of your questions:
You aren't (and should not) build up against the dirt, this is a misconception and/or or a very poor practice that will lead to the wall buckling and failing. You will have either a filter cloth laid down and/or a earth plaster barrier and gravel backfill with additional tile or rock drains, and the stone is laid of of this. The better walls (even though they use more stone) are laid like retaining walls. I also do not recommend square corners for the novice stone mason, not do I even use them myself typically.
How do you build a stone wall that will back up against dirt (basement), without using cement?
If the wall is built properly, (and you're not over a high water table or flood zone) this will not happen even if you dry laid the stone and use no mortar (unmortared wall are not generally recommended for basements if you are not a master dry laid stone mason.) Even if water came through, if all is built well, it would drain away immediately.
But that doesn't have tiny gaps where rain-saturated dirt outside can leak through?
The simplest answer to this is cob, then you are going to point with traditional lime, both of which has been done for millennia. The cob mortar in this case, as you are not a mason by trade, can be lime stabilized. DO NOT, NOT LET ANYONE TALK YOU INTO USING PORTLAND MORTARS OR CONCRETES.
We would like to have a basement under our cob home, continuing the stone that is the first 2ft of the cob house (foundation walls). But what in the world do you put between the stones?
You may not see it, but that is how it is done, and you need to master those skills to do it, or hire someone that has them.
I don't see bare stacked stone as a good basement wall.
In this case it is not touching the ground it is touching the stone and the stone is being used as your "grade barrier" between the cob wall and the earth. Drainage is your key focus here and the foundational element to enduring architecture.
I don't want to use portland cement, since the stone wall will touch the cob wall (and I hear that wicks moisture badly). But cob can't touch the ground, so I can't use clay/dirt between the stones as it backs up to the ground.
Try thousand of years, and I described that above already I think.
I know people have been making stone basements (especially for storing potatoes, etc) for hundreds of years, but what did they hold it together with; that didn't leak water when it rained hard??
We really need you to upgrade your profile so we all know where you are at, and have an idea of you building environment.
If you are in a predominantly dry area then a basement can be good for many reasons. If your in an area that gets a lot of rain with clay soil I think basements are a giant waste of resources; mold, dampness, cold and unwelcoming and you'll spend a massive amount of energy to keep it livable. I'm a huge fan of just laying a foundation on the ground rather then in it.
Many of the building skills Jay mentioned are way beyond most people, even if you can do the work yourself its a lot of energy for little gain. The earth in a funny think it likes to provide you shelter opportunities above but rarely does it offer those same opportunities underground.
If you're just starting out and haven't got much experience then take a project that you can handle. If you've never built a building (specifically one a person can live in) then you should really scale back your expectations cause its a huge process even if you're just hiring trades to do the work. The learning curve can be extreme and you have to remember that a home is by definition safe and healthy, people have known this for thousands of years.
I would recommend not having a basement and use those resources in an area of the home that will provide you more return on investment, like more square footage.
Also at the very least get yourself an engineer, last thing you want is to put your heart and soul into building a home that falls down or sucks up a huge amount of maintenance energy.