I know this is going to come up.
Somebody is doing the free program and they submit a
BB. The evaluator sees it and it falls short and says "not yet" or "keep going" or "do it over." And the student says "why?" or wants to question the evaluation.
Over at coderanch, the beginning of the site was "the java college" which was later renamed to "the
cattle drive". There were 8 assignments. The idea was that an assignment would be sent in to the nitpicker and you would have to do it over and over until the nitpicker could find no further nits to pick. In time, it became so popular that we had to start charging $200 per student just to thin the herd a bit.
Most students seemed to really like this process. But about 1 in 4 would argue with the nitpicker and basically try to change the program or attempt to move on to the next assignment by being argumentative.
So what we did was we set up two price structures: $200 and the nitpicker is always right. $2000 and the student is always right. So when this sort of arguing cropped up, we would say "No problem - just send over that extra $1800 and we will get this all sorted out for you." It worked 100% of the time.
Of
course, it is possible that it won't work here on permies, but .... it could be worth considering.
I do think there will be times where a person consistently does a really poor job and will need some extra guidance. And that individual is especially difficult/challenging. I suppose that that student will need to hire a tutor or pay an evaluator to explain in enormous detail.
So as much as I wish to set this framework up to be free for everybody, I do think "everybody" needs a bit of a qualification: students that appreciate and respect the
gift of evaluation by the evaluators.