For soy, have a look at some of the work from the Weston Price Foundation who have some very interesting articles about soy.
Myths and Truth about Soy and
Soy Allert being good starter articles. There are also some interesting tidbits in
Nourishing Traditions by Fallon, and if memory serves, some even more interesting bits (with nice citation to the original sources) in Michael Pollan's Omnivore's Dilemma. Soy being quite the political quagmire, I've found surprisingly few studies on humans that show
strongly in favour or against consuming it in unfermented form.
Broccoli is more difficult. For me, grocery store broccoli makes me feel really groggy after eating. If I have more than a few bites on my plate, I start to feel all vomity. Home grown broccoli (mostly rab really) doesn't make me feel anywhere near as bad when I eat it, but then I get those annoying cabbage moths and earwigs. (what's worse than finding a worm on your plate? Finding half a worm.)
A sudden addition, subtraction or change in the amount of broccoli you consume can be quite bad if you are on blood thinners, or have a condition that requires managing your blood 'thickness'.
Grain, pulses, fruits, veg, anything high in fibre, if I eat a lot of it, then I get quite sick and hospitalized. Certain conditions like Crohn's and other digestive issues can react very negatively to high fibre in the diet. But a little bit of fibre in the diet isn't so bad. So maybe half a cup of whole grains or pulses (dry amount) per week is okay for me, but that much a day... no way!
Then again, it may be more than just the fibre. If I make a sourdough bread from whole grains, fermenting the grain for at least 24 hours prior to baking, then I can eat a whole whack of the stuff without feeling ill. Then again, I'm using whole grains that I've milled just before starting to ferment them. I've had friends who are (edit to add, diagnosed) EXTREMELY gluten sensitive be able to eat my wheat sourdough bread with no symptoms at all. It's so hard to tell what's actually causing the problem.
I think, nutritionist science currently lacks the skills to individualize the 'good for you' criteria to the individual person, enough so that it is actually good for you. Like the OP mentions, there is so much
energy and expense put into educating people and professionals in favour of specific hype. I highly recommend the two
books I mentioned above to anyone interested in food and nutrition. They aren't perfect books, in fact I disagree with a good chunk out of each of them, but they are excellent gateway books where you can find a topic you are interested in and follow the citations to learn about it in more depth.
Another interesting book which has had a lot of influence on nutritional opinion is Diet for a Small Planet by someone with a name I don't remember. The stance is hugely pro-soy, but I think the author(s?) was working with limited information on the subject. The main goal of the author was to reduce the gluttony of meat consumption in the US, or at least to stop feeding meat animals food crops that can be used for human consumption. Most interesting thing in this book for me, was the idea that the nutrition in the food is not the same as the nutrition absorbed by the person eating it. By combining certain foods, your body gets a much larger amount of nutrition then it
wood eating each one individually - sometimes more than the sum of the individual foods contain. Or so the author tells us.
This is going to sound a bit like a conspiracy nut, however, I've noticed that the results google comes up when searching for information about foods (especially foods with powerful political lobbies like Soy) has changed DRASTICALLY in the last few years. A few years ago when I discovered I had an latent onset sensitivity (igg immune response) to soy there were a great many pages both pro and anti soy, available with reference to academic and other supporting sources. Now I do the same searches, and google provides results that are heavily in favour of one side of the debate with fewer references to their sources.
My favourite place for information nowadays is the library. Of
course books are somewhat subject to censorship, the authors each have their own point of view, and I'm certain one day the library may have to adjust their content depending on where their funding comes from. But for now, most books cite sources which I can then look up and read the original studies (if it interests me). At the very least, I can look up to see if they are citing a real source of information, or just making up evidence to support the author's point of view.
Edit to add: Can't find it now, but there is a great moment in the Simpsons where broccoli kills Homer and the Dr. talks about how broccoli is the most deadly of all vegetables. That clip would be a fun addition to this thread.