According to a terse statement issued yesterday by the president's office, Sefue will be deployed to 'another position'.
He is replaced by Tanzania's ambassador to India, John Kijazi, who served as permanent secretary in the Ministry of Works when Magufuli headed that portfolio in former president Jakaya Kikwete's administration.
The chief secretary doubles as head of the civil service and secretary to the cabinet. The decision to remove Sefue comes as something of a surprise given that Magufuli confirmed him to the post on December 30 when he made appointments of permanent secretaries.
Since Magufuli took over the presidency in November last year, Sefue has been at the forefront in articulating his ‘hapa ni kazi tu’ policy, describing it as being "all about delivery."
He was also often called upon to address the media when Magufuli sacked or suspended top government officials over corruption allegations.
It was Ambassador Sefue who faced the media and highlighted the early successes of the Magufuli administration during last month’s marking of its first 100 days.
A career diplomat, the 61-year old served as Tanzania's ambassador to Canada from 2005 to 2007, was briefly the country's envoy to the United States (2007), and then permanent representative to the United Nations.
Sefue also previously worked as speech writer for and personal assistant to then-president Benjamin Mkapa.
He was appointed chief secretary by President Kikwete in December 2011, a post he has held for four years and three months.
He has a masters of arts degree in public policy and administration from the Institute of Social Studies (ISS) at The Hague, and also holds a post-graduate diploma in international relations and diplomacy from the Centre for Foreign Relations in Dar es Salaam.