I think that 'how small can you go' partially depends on climate.
In a warm climate, with mild winters, you can go pretty tiny.
A nice patio, with BBQ, table/chairs, maybe a small storage shed or lockers, etc.
You don't go inside until after dark. You enjoy your morning
coffee out there.
Washer/Dryer, refrigerator and many other things can be located on a covered deck.
I have even seen sofas and TVs kept outdoors year round (covered of
course).
In a colder climate, with short growing seasons, you can only spend so much time in the outdoors space. In extreme climates, with long, bitter winters, even an 'average sized' house 'grows in' on you. Cabin Fever starts setting in long before spring has sprung. You need more storage space to accommodate bulky winter ware, extra blankets, etc.
In most areas, the tax man hits you a lot less per square foot for covered decks than he does for the enclosed house area. I know a couple who have a cement patio with +/- 6'
wood fence surrounding it, and a translucent roof. It is almost the size of the house itself, yet only gets charged about 10% of what the house does. They spend almost all of their time "on the patio".