Sounds strange. Able to control via HomeAssistant but failed to control with phone? Did you open VPN service on your phone? Make sure your phone is connected to same 2.4G network.

No VPN in use, I am definitely connected to the same network. I've been able to replicate this twice by factory resetting.

rather strange, only possible explanation is the phone had lost connection to the Bulb, thus the disabling action of the remote control did not actually take effect.
would like to suggest the following:
1, reset and reconfigure
2, once you are able to contact the bulb locally and remotely, marked down the IP of the bulb.
3, try and turn off the remote control function.
4, check again see if you can still control the bulb locally, if not, try to ping the bulb from your phone and a PC respectively.

matt - I have static DHCP reservations for the light bulbs so they always get the same IP address assigned.

parisweb - there is no VPN in the path here. All on the same LAN subnet.

Your bulb has the latest firmware? If this keeps happening, I think it's better contact tplink support directly.