@Will307 i've done this over 10 times. My smartthings does the same thing. It pulls in deleted devices.

I moved to a new house. I removed TPlink switches that stayed in my old location and added new ones. Everything works fine accept Alexa discovery still finds my old devices even though I removed them...