How, exactly, do I connect my 2nd router (D-Link DI-624) to my first

How, exactly, do I connect my second router (D-Link DI-624 802.11g Wireless 108 Mbps 4-port) to my existing router (Linksys BEFSR41 Etherfast Cable/DSL Router)?
I have read conflicting advice.

Current set-up:
I have 3 computers in the basement (all hard wired).
I have 3 computers on the first floor (2 hard wired and one wireless but the wireless one is not functional (missing parts) yet).
I have 3 computers on the second floor. There are zero wired ports on the second floor so they will all need to connect wireless.

Just switched to a new cable modem.
Attached to the cable modem is a Linksys BEFSR41 Etherfast Cable/DSL Router.
Attached to Linksys BEFSR41 Etherfast Cable/DSL Router is a Linksys EZXS88W 10/100 8-port switch.
Attached to the Linksys EZXS88W 10/100 8-port switch are 5 computers and (hopefully soon) a D-Link DI-624 802.11g Wireless 108 Mbps 4-port Router (but on a different floor of the house).

The cable feed (and cable modem) comes into the house in the basement.
The first router (Linksys BEFSR41 Etherfast Cable/DSL Router) is set up in the basement next to the cable modem. I need a switch as my Linksys router only has 3 ports plus the uplink.
The switch (Linksys EZXS88W 10/100 8-port) is in the basement next to the Linksys router. I need the D-Link (wireless) router as I have 3 computers on the 2nd floor where there are no jacks.
Even though it has more ports, the (wireless) D-Link router cannot act as the sole router as it still does not have enough ports and more importantly must be installed on the first floor as the signal is too weak to establish a strong connection from the area of the basement where the other equipment is to where the computers are on the second floor.

What do I need to change on my first (Linksys) router?
I am pretty sure that I need to assign a static IP to the second router from the first router. But how do I do that? Unfortunately, my current (most up-to-date) firmware does not match the screenshots/descriptions in the manual that I have. Nor does the firmware actually match the screenshots/descriptions in the manual that is online at Linksys (they are all close but there are several fields that are different).
I am pretty sure that this router needs to be set as "Gateway" (rather than "Router" ) and I even see how to do that (but is this correct?).
Do I need to connect the uplink port from this router to the port on the switch to which the second (D-Link) router is attached? And/or Do I need to connect the uplink port from this router to the WAN port on the second (D-Link) router (totally bypassing the switch)?
Do I need to use TX, RX, RIP1, RIP2?
What else do I need to set for this router?

What do I need to change on my second (D-Link) router?
I am pretty sure that I need to turn off DHCP on the D-Link but how do I do that? I think that the factory settings come with it on and as soon as I connect the second router I will have conflicts wont I?
Do I need to use the WAN port on this router or can I use just any old port?
Do I need any cross-over cables (I don't think so but just wanted to confirm)?

The easiest way is to attach one of the LAN ports on the second router and disable it's internal DHCP server. In this setup it should bridge WiFi to the Linksys which takes care of the rest such as routinh to the internet and IP address assignment for the wireless clients as well as the wired.