I have an old Magnavox tv with Zenith converter box and a Magnavox DVD player but no remotes for the tv or DVD player. The audio works but not the video.
Check that the video out of the DVD is connected to the correct input on the TV. The fact you have a OVA digital converter box should not matter unless you don't have enough video inputs on the TV.