For harvesting leaves, mid-morning is usually considered best.
You want to wait late
enough for the dew to dry, and allow the warmth to draw the essential oils into the leaf.
If you wait too long, the photosynthesis and heat will begin exuding these oils, which can cause them to either evaporate, or turn bitter on the leaves surface. Mid-day is too late for maximum potency, and optimal flavor.
For harvesting seeds, you want them to mature (but not shatter)
on the plant if possible. With annuals, if you expect poor weather before the seeds are fully mature, you can pull the whole plant up (saving as much
root as possible), and hang it upside down over a tray in a warm, dry place, allowing the seeds to further mature, andfall into the tray.