Most likely that connection on the tv is line level, which has no power in it. Without a power source, you won't hear anything from those speakers. You still need some sort of speaker amplification for that setup to work. I had a TV in the 80s that had speaker outputs on it, but they were still push pin style speaker jacks. A few old amplifiers did have an rca style speaker terminal on them. Those inputs/outputs are fairly sensitive in newer tvs, there is a small chance you could damage the input or output stage by plugging a speaker directly into an rca jack.