Need help identifying tree

Hi everyone! I'm in the California Bay area in a semi costal area. This tree is dropping a dark purple, almost olive colored fruit that the local racoons and squirrels are going crazy for. I thought it was some kind of chokecherry or ornamental plum but upon closer inspection of the fruit, I am unsure. The fruit start green, turn to bright red and then reach a dark olive color, at which point they start to drop. if I can remember clearly, I think it had clusters of white flowers in the spring. Does anybody know what it is? I truly appreciate any and all input. Thanks!

Could it be False Olive?  http://suntrees.co.za/buddleja-saligna-false-olive-witolien/

Tyler Ludens wrote:Could it be False Olive?  http://suntrees.co.za/buddleja-saligna-false-olive-witolien/

That was my first guess when I first saw the fruit, but the leaves are all wrong. They are large, semi glossy and the same shade of green and general shape as most citus leaves.

Is prunus illicifolia a possibility? There are two varieties, the one from the channel islands commonly escapes from cultivation so it could be either if so. The drupes don't look to have much flesh on them.