Meles Zenawi gained power in 1991 and went on to become a towering political figure on the African continent who was widely credited for steering one of the world's poorest countries to high economic growth. He died overnight in a Brussels hospital after a battle with illness.
Speculation that Meles, an ally of Washington who twice sent troops into neighboring Somalia to help crush rebellions, was seriously ill had grown after he failed to attend an African Union summit in Addis Ababa last month.
State media said Deputy Prime Minister Hailemariam Desalegn would be sworn in as acting prime minister by parliament.
"The prime minister passed away around midnight. It's time for his remains to come back," government spokesman and Meles' right-hand-man, Bereket Simon, told reporters on Tuesday.
Bereket said Meles had been ill for a year and was recuperating before being suddenly rushed to intensive care.
Meles seized power in 1991 from Mengistu Haile Mariam's military junta and went on to become a towering political figure on the continent who was widely credited for steering one of the world's poorest countries to high economic growth.
Although rights groups criticized him for cracking down hard on dissent, the West generally turned a blind eye to the repression, reluctant to pick a fight with a partner in the fight against al Qaeda-linked groups in Africa.
British Prime Minister David Cameron hailed Meles as an "inspirational spokesman for Africa."
"His personal contribution to Ethiopia's development, in particular by lifting millions of Ethiopians out of poverty, has set an example for the region," Cameron said.
The secretive government refused to reveal where he was being treated or the nature of his illness but a European Union source said he died in Brussels where he had been a patient at the Saint-Luc University Hospital.
Somalia's al Shabaab militants, who encountered Ethiopian troops twice under Meles' tenure, once in 2006-2009 and again from Dec. 2011, hailed the former leader's death.
"He led the African leaders who had fingers in Somalia for two decades, but all in vain," spokesman Sheikh Ali Mohamud Rage told Reuters.
Bereket, who declared a period of national mourning until Meles' body returns, said Africa's second most populous nation was stable and would continue on the path charted by Meles.
The ruling party, Ethiopian People's Revolutionary Democratic Front, will meet to select a permanent successor, although a date has not yet been set.
Negasso Gidada, a former president during Meles' tenure and now chairman of the opposition Unity for Democracy and Justice, said he hoped the succession of power would be peaceful.
"We urge the EPRDF to change for the good the political, democratic and human rights situation in the country," he said.
David Shinn, a former U.S. ambassador to Ethiopia, said he expected many of Meles' defence policies to remain the same.
"For internal security reasons, there will be a continuing focus on Somalia and I do not foresee any significant change towards Eritrea," said Shinn, referring to Ethiopia's arch-foe with whom it fought a decade-long border war.
Meles presided over a seven-year run of double digit economic growth, advocating a mix of heavy state spending and private investment.
He was widely applauded for ploughing money into infrastructure but criticised by some for selling off swathes of land to foreigners. Many Ethiopians complain that his close business ties with China did not translate into more jobs.
International rights groups say Meles was intolerant of dissent. He rounded up numerous opposition leaders after the disputed 2005 polls and several opponents and journalists have been arrested under a 2009 anti-terrorism law.
"Today is a day of joy for most Ethiopians and all freedom loving people around the world," opposition website Ethiopian Review said, describing Meles as a "genocidal tyrant."
The European Commission said Meles had worked hard on African unity, climate change and development, but hoped the country would improve its human rights record.
"I sincerely hope that Ethiopia will enhance its path of democratisation, upholding of human rights and prosperity for its people, and of further regional stabilisation and integration," European Commission President Jose Manuel Barroso said in a statement.
State television said details of Meles' state funeral would be announced soon.
Acting Prime Minister Hailemariam was an advisor to Meles in 2006 before being picked as his deputy in 2010, a surprise because of his relatively young age.
He had also replaced Meles as chair of a number of parliamentary committees in the past few years, a sign that he was being groomed for the post, diplomats say.