Israeli security forces were seen dragging a worshipper, while another police officer appeared to punch a Palestinian man during the confrontation on Wednesday.
Stun grenades were fired as police forces tried to disperse the crowd, while several people were seen throwing objects in response, said Al Jazeera's Harry Fawcett, reporting from the scene of the clashes in East Jerusalem.
"We have another (round) of violence here right in front of al-Aqsa compound. This shows you how sensitive this period is," he said.
Security forces have cleared the area outside al-Aqsa compound, ordering vehicles out, and the situation have since calmed down, he added.
The confrontation followed clashes between Israeli security forces and protesters after evening prayers in the occupied city on Tuesday.
President Mahmoud Abbas' Fatah party has called for a Day of Rage after metal detectors and turnstiles were installed at the entrance of al-Aqsa following Friday's deadly shoot-out that left two Israeli security officers and three Israeli Palestinians dead.
In another development on Wednesday, Jerusalem's top Muslim leader called on all of the city's mosques to be closed on Friday in protest against the new Israeli security measures.
Grand Mufti Muhammad Hussein said Palestinian Muslims should all gather outside the gates of al-Aqsa Mosque for the weekly Friday prayer.
Hamas and Islamic Jihad - two Palestinian groups - have also called for protests on Wednesday in the West Bank and Gaza Strip.
Sheikh Azzam Khatib Tamimi, head of the Waqf, the Islamic authority in charge of al-Aqsa, said earlier on Wednesday that the authority had instructed "imams in all the mosques in the city of Jerusalem not to hold prayers in their mosques and to come to al-Aqsa".
The Waqf indicated in a statement on Wednesday that the longer Israel delays the removal of the metal detectors, the worse it is going to get.
Palestinians have already been performing prayers outside the compound in protest since it was reopened on Sunday after a two-day closure.
Al Jazeera's Fawcett said the protests are not just about the implementation of the new security measures by Israel.
"This isn't about one isolated installation of a security measure. For Palestinians this is about the fact that Israeli forces are an occupying force and the potential that Israel is in contravention of the UN articles guaranteeing freedom of worship, guaranteeing that occupying forces do not change any part of the status quo within the city," he said.
Also on Wednesday, Israeli authorities closed the compound to Jewish visitors after what they descrfibed as a violation by some Jewish individuals who appeared to have prayed at the site, which is forbidden to non-Muslim worshippers.
Under the status quo, Muslims are given religious control over the compound and Jews are allowed to visit but not pray there.
On Friday, Benjamin Netanyahu, Israel's prime minister, said he did not want to alter the status quo, but Palestinians fear Israel is trying to retake control of the site by stealth.
The site houses the al-Aqsa Mosque and the Dome of the Rock shrine, Islam's third holiest site after Mecca and Medina, as well as the ruins of the Biblical Jewish Temple.