fine wooden box repair, the easy or the hard(er) way?

We picked up this very nice wooden box at an auction.(<$5) It came with some junky knives that went the metal recycling.
It had some horrible fake velvet and foam that was glued to the top and bottom of it and the bottoms of both of the nice trays it came with.
After careful removing, scraping and cleanup,  (outside, with a mask on) there is just a bit of glue residue left.

I know the proper thing to do would be to sand it all away but I just don't think I'm that invested.
I plan to use it for storage of some fine art tools so it doesn't need perfection or full food grade.
I am leaning towards using some of my rabbit skin glue and just putting some leather panels over the old glue. (I have a good stash of reclaimed leather)

I'm trying to embrace the concept of "done is better than perfect" more but want to make sure there isn't some big potential issues I'm missing by taking the easy way out.

I would wash the inside with Murphy's Oil Soap.

Then when dry use an oil of you choice to finish it.

the oil I would used would be furniture polish because I have that.

This is what it looks after the oil soap wash. I haven't done any oiling yet since I am considering the leather and didn't want to risk interfering with the new glue adhesion.