"We conclude that the assembly did not hold the president to account", said Chris Jafta, a Constitutional Court judge who read out the ruling.
The court found using an ad hoc committee to hold Zuma accountable in parliament was not acceptable because it could easily be interfered with.
He found applicants EFF‚ UDM and Cope were themselves part of a process that already taking place to make rules for motions to remove the president under Section 89 of the Constitution.
"The constitutional delinquent, Mr Zuma, in no time will be scrutinised, will be brought before Parliament to be accountable because his own political party has failed to hold him accountable".
The Constitutional Court's ruling followed its conclusion a year ago that Zuma violated the constitution when he benefited inappropriately from state funding for his Nkandla home.
In March 2016, the court ruled that Zuma pay back some of the roughly 15 million dollars in state money spent on upgrading his private home.
Holomisa said that while he remains hopeful for a favourable judgment, he does not believe the African National Congress (ANC) Members of Parliament (MPs) will impeach Zuma.
"Properly interpreted, Section 89 implicitly imposes an obligation on the assembly to make rules specially tailored for the removal of the president from office".
The unanimous ruling by the 11-judge Constitutional Court, a central pillar of the democracy established the fact that Zuma had failed to "uphold, defend and respect" the Constitution by ignoring the findings of the watchdog led by former Public Protector Thuli Madonsela.
The parties wanted the court to compel Speaker Baleka Mbete to initiate impeachment proceedings against Zuma.
Chief Justice Mogoeng Mogoeng wrote a minority judgment saying the majority order was a "textbook case of judicial overreach" into Parliament. While this was a majority judgment, the Concourt Chief Justice Mogoeng Mogoeng and his deputy Raymond Zondo disagreed. Similar views were expressed by Cope leader Mosiuoa Lekota.
He said the judgment means that in future, "nobody, but nobody can abuse power at will and protect each other while violating the law of the land". EFF secretary-general Godrich Gardee said his party welcomes the court's ruling and that the ANC had to implement the court order.