Orange tree grown indoors -- leaves turning yellow from the bottom

I have an orange tree grown from seed that I keep indoors. It appears to be growing well from the top but the leaves at the bottom are slowing turning yellow (beginning at the bottom). How concerned should I be? And what should I do differently?

I have a bunch of indoor citrus and yours looks great!  Sometimes if the leaves don't get sunlight or get a little dirty they can die back, but that's just part of growing up.