The writer on me say perhaps the reason you can teleport with swords and armor made of metal, but not the raw metal is because the equipment is specially treated during the forging process to be teleport-friendly.

But than you find you can teleport gold coins and iron nails, but can't do it with dragon eggs and my theory start  to go wack. There is no rhyme or reason to it outside trying to make mining harder.