The only difference between a quarry and a mine is what comes out of it; besides that, they're identical in function. A tunnel into the earth, reinforced with support beams, to extract a material.
If you've heard of, and/or seen images of, a strip mine then think about that for a second. In order for the miners to get at the precious metals or gems they're looking for, they need to remove tons of dirt and various kinds of stone. Does all that dirt and stone just get thrown away? No, it gets carted off somewhere to be used. It may be called a mine, since the intended function is the collection of precious metals, gems, and the like, but that doesn't mean it isn't also a quarry.
Basically, I think the model is fine the way it is, there's nothing wrong with it at all.