You can build a 20 m2 building in France without architect
plans and major planning permission. However, you will still need some permit of some kind, even if it is just a "declaration
de travaux" and probably only if it is permitted to build on this particular piece of land. Of course, all will depend on what your situation is and will vary from place to place. Inevitably you will have to deal with the dreaded French bureaucracy - no escape there!
Mountainous regions, especially in high pasture lands have their own laws and unless the land you buy already has a CU (certificat d'urbanisme) i.e sold with and existing building permit, you will find it quite impossible to acquire one. There is rarely a building permit given. If the land has a building on it already like an old barn or some such, then it is slightly different, although you still have to go by the "loi de la montagne" regarding abandoned and crumbling barns and their usage. It will also, in some cases, very much depends on the Maire of the commune in which the land is.
Speaking French will be a tremendous advantage. And if you are looking for a mountainous region, I would suggest that the Pyrenees will be slightly more accommodating than the Alps.