It depends on what you find most important in your smartphone. First off, if you are used to android and have never used an apple device before, I see no reason why you should make the switch. Second, if we are looking at just short term consequences of this exchange, the note 3 will be light years ahead of the 4s in almost every, if not all performance aspects. Third, if you are inclined to the large screen of the note 3, all the iphone screens will seem dwarfish in comparison.
But that is not to say switching is a terrible idea. If you keep your note 3 for 3 years, by then, the iphone you would be given would most likely be light years ahead as well. Also, if rumors are true, apple will be coming out with a larger version of the iphone which could negate my third point about screen size.
The point is, switch if you want to have better in the long run, or stay if you don't want to deal with small screens, slower performance, and the need to relearn an OS in the first year or so.