I've become quite suspicious that some of the seeds, both commercially produced or saved from fruit and veg from the shop/supermarket have had hardiness bred out of them, and some even like canola, been genetically modified so that saved seed won't ever grow.
I did manage to grow capsicum from a very ripe red fruit, and it over-wintered in the garden - second year fruits really small. Broccolini produced seed from purchased seedlings, but then no action when they were sown the following year, although the plant did grow again from the stalk I'd left in the ground. Same with the silver beet (Swiss chard) - practically a forest of that. It doesn't do well once the weather warms up.