@sixhead 

Only my phone is connected to the access point, even though there are other devices right next to it that are still connected to the router instead of the more convenient access point. I suppose it's because I used my phone to set up the access point.

My phone doesn't automatically connect to the router when I'm on the floor that it's on, and just reconnecting it to the WiFi doesn't work. 

I'm suspecting that the devices on the network connect to either the router or the access point, and don't switch over the two. How can I solve this issue?

In my opinion, this is about roaming feature, which is the behavior on the wireless clients side. If your phones cannot automatically switch wifi between, it might be better to use different SSIDs on the router and AP to distingush them.