Read more: US Navy Seal dies during skydiving show over Hudson River
Two of the passengers were pronounced dead at the scene, while the other three were taken to hospital in critical condition. They were pronounced dead in the early hours of Monday.
Police said a sixth person, the pilot, managed to free himself from the helicopter after it hit the water. He was rescued and taken to a hospital, before being released in good condition.
Federal investigators have been dispatched to New York to determine what caused the crash. Fourteen officials from the National Transportation Safety Board are expected to arrive on Monday.
Read more: Four survive helicopter crash in Grand Canyon
Video footage of the crash showed the red helicopter slamming into the water near Manhattan's Upper East Side neighborhood before turning on its side with the rotor blades still turning.
According to the New York Daily News, the pilot sent out a distress call, saying the engine had failed. Authorities said the aircraft went down just after 7 p.m. local time (2300 UTC).
Victims submerged 15 meters into East RiverNew York Fire Department Chief Daniel Nigro told reporters that rescue divers had to dive 15 meters down into the East River to rescue the passengers, who were stuck in the aircraft at temperatures near freezing.
"We are told the five people were all tied tightly in harnesses that had to be cut and removed," Nigro said. "It took a while for the divers to get these people out. They worked very quickly as fast as they could. It was a great tragedy that we had here."
Authorities said the helicopter was a private charter taking photos of New York City. The sightseeing service that owned the helicopter, Liberty, organizes aerial tours of the city.
There was no indication of what could have caused the crash. The weather in New York City was calm with clear skies at the time of the accident. The US Federal Aviation Administration said it was investigating the accident.
Crashes in New YorkWith the skies over Manhattan constantly abuzz with tourist helicopters, private aircrafts and traffic reporters, crashes in the area are not unheard of.
In 2009, a sightseeing helicopter operated by the same company collided with a private plane and crashed into New York's Hudson River. Nine people died in the accident, including a group of Italian tourists.
In October 2011, a helicopter crash in the East River killed a British woman who was visiting New York City for her 40th birthday.
In July 2005, two choppers crashed into the East River in the same week, although there were no casualties in either incident.
dm/ng (AP, AFP, dpa)