The plane had just left the city’s La Guardia airport when it is thought to have hit a flock of birds, which disabled two engines, said authorities.
The Airbus A320, with 148 passengers and five crew, was destined for Charlotte airport in North Carolina when the accident happened.
There are reports of some injuries – but everyone is believed to have survived.
Passengers said the captain told them to brace for impact and then the plane came down.
Emergency services then tried to get the passengers off the aircraft as quickly as possible.
TV footage showed dozens of them on the wings of the plane as it floated on the freezing cold river.
They were later all rescued. One passenger said he thought the plane had dropped about 100ft and he had stood in about 5ft of water after the crash.
The jet took off at 3:26pm local time (8:26pm UK time) and was in the air for just three minutes.
Witnesses on the ground reportedly saw a bird strike.
Passenger Alberto Panero said: “All of a sudden, the captain came on and said, ‘Brace for impact,’ and that’s when we knew we were going down, into the water.
“And we just hit and somehow the plane, you know, stayed afloat and we were all able to get on the raft. It’s just incredible right now that everybody’s still alive.”
The FBI said it had no information that the New York crash was an act of terrorism.
Department of Homeland Security spokeswoman Laura Keehner said: “There is no information at this time to indicate that this is a security-related incident.
“We continue to closely monitor the situation, which at present is focused on search and rescue.”
One witness Barbara Sambriski said: “I just thought, ‘Why is it so low?’ And, splash, it hit the water.”
Emergency services reportedly had stretchers waiting at the riverbank to take people to hospital.
Former pilot Eric Moody told Sky News: “It has to be big birds to stop the engines.
“If you go through a flock of big birds it can be quite disastrous, as you can see.
“I would say that was a very successful ditching of that aircraft, if all those people got out safely.
“Whoever has flown it has done a very good job.”
Skynews