As a source for already growing garden herbs consider the supermarket. The local big chain store near me sells herbs in small flower pots: parsley, basil, that sort of thing. Like seemingly everything else in the super market these days, these have a "Best Before" or "Sell by Date." Once that has passed you can find them in the section with all the other sad produce for a very low price -- pennies instead of pounds (yes, I am living in the UK).
Take a sad and sorry little pot of parsley home, put it in a bigger bucket, give it a bit of water and it will be looking fine in no time. The last one I did this with lasted a year or two.
I have also grown tomato plants from the odd mouldy cherry tomato. No, I didn't get a huge yield, but my aim was simply to have something green and growing in the flat. And that lovely smell whenever you handle a tomato plant! It can't be beat!
