According to the screenshot of your configuration, it seems all good...
Connecting directly to your router has no such a problem? IP address on all your wireless clients are obtained automatically or some of them are using static IP?
Is using the same SSID typically better / more advised, or different SSID?
Doesn't matter.
What about static / dynamic IP of the extender?
Dynamic IP is OK.
Does the IP address of the extender NEED to fall within the assignable IP range of the main router?
If you use Dynamic IP, extender's IP will be assigned by the main router's DHCP range.
DHCP server of the extender is definitely off, yes?
It should be off.
Who assigns the IP addresses for devices connected to the extender?
The main router is the DHCP server which assigns IP address for all devices in the network.
Do I need a fireware upgrade?
What's the firmware version of your extender? If it is not the latest version, you can upgrade it first.

My firmware version is as follows. Is it the most updated? If not how can I update it?

Firmware Version: 3.14.8 Build 140827 Rel.34945n


None of my devices are on static IP. Should I try to assign static IPs to everything to try to avoid this problem?

140827 is the latest version currently...
On your main router, you didn't configure address reservation as well?
That's weird... If it is convenient for you to assign static IP for all clients, you may give a try.