The flavor of the kernel is fine, similar to an almond after it's properly cured, dried, and/or roasted. However, the shells of the pit are so thick and hard to crack that they really aren't worth the effort. They are one of the toughest nuts to crack I've encountered. The only way is with a hammer, since they are too small for my mechanical nut cracker, and when they do finally open, the force required tends to be enough to smash the kernel into a paste of small pieces and shell particles.
I have had the same experience with my Hall's Hardy Almond tree (which is a peach-almond hybrid, actually). The nuts are actually very tasty (they are high in prussic acid and MUST be boiled before roasting), but they are just so hard to shell that I haven't bothered picking them up for years. I've noted that even the squirrels must think so, because they never bother with them until late into the winter, when the other food sources are running low. Then, they will go after them and clean them all up.