According to trade union leaders and analysts, the ongoing purge by the new government is already creating an atmosphere of uncertainty among public leaders and senior civil servants, which could in turn affect their efficiency.
President Magufuli on Sunday removed Chief Secretary Ombeni Sefue with immediate effect, barely two months after confirming him to the position.
The president yesterday swore-in John Kijazi, Tanzania's former ambassador to India, as Sefue's replacement. It remains unclear why Ambassador Sefue was shown the door, but the president's office said the former CS will be assigned other duties.
Photos from State House yesterday showed an expression-less Sefue as he witnessed the swearing-in of his predecessor and posed for group photos with senior government officials afterwards.
Before being swiftly elbowed out of Magufuli's inner circle, Sefue was seen as the de facto spokesman and chief policy articulator of the new government.
Magufuli has purged several senior figures from his government, citing corruption allegations or the need to improve efficiency.
Top civil servants who have already faced the axe under Magufuli's administration include the former Commissioner General of the Tanzania Revenue Authority (TRA), Rished Bade, the former director general of the Prevention and Combating of Corruption Bureau (PCCB), Dr Edward Hoseah, and the ex-Tanzania Ports Authority (TPA) chief, Awadh Massawe.
The immediate former director general of the country's biggest pensions fund, NSSF, Dr Ramadhan Dau, was also a recent casualty of the government purge.
However, observers say the ongoing dismissals of top civil servants could end up being counter-productive if not properly executed.
"Arbitrary dismissals of civil servants can create an atmosphere of uncertainty in government because people don't know what is going to happen tomorrow and if they will be next in line for the chop," Nicholas Mgaya, secretary general of the Trade Union Congress of Tanzania (TUCTA) told The Guardian.
"This situation of uncertainty in security of tenure causes people to work in fear, hence it could affect the efficiency of the civil service."
Mgaya warned that the Magufuli government could face expensive lawsuits of unfair dismissals if it continues to sack public officials without following procedure.
"The laws are very clear about what needs to be done when taking disciplinary action against civil servants. In some cases such as the dismissal of the Mwanza regional administrative secretary (RAS), these procedures have been flouted by the government,” he said.
President Magufuli in January sacked the Mwanza RAS, Faisal Issa, for indiscipline after he allegedly had an altercation with his boss, Mwanza Regional Commissioner Magesa Mulongo.
Mgaya said it was wrong for the president to sack the RAS the same day after receiving complaints against him without giving the official the right of reply to show cause why disciplinary action should not be taken against him.
He warned that some cabinet ministers, regional commissioners and district commissioners could fall in the trap of arbitrary dismissal of public officials to try to impress the president.
Some analysts said the jobs of cabinet ministers themselves are not safe after Magufuli threatened to sack any minister who go against his wishes.
The president last month said he would dismiss cabinet ministers who failed to declare their assets and liabilities and sign an integrity pledge.
The Tanzania Teachers' Union (TTU) acting Secretary General, Ezekiah Olouch, similarly warned that the dismissal of some public officials appeared not to follow procedures, especially at district and regional levels. He said in some incidents, public servants have been suspended by regional and district commissioners, who have no jurisdiction to do so.
Olouch said the government should be extra careful and conduct the clean-up exercise in a way that protects the dignity of employees.
Prof. Mwesiga Baregu, a senior lecturer at St. Augustine University, said the performance of civil servants especially decision-makers will be affected by the anxiety and perceived lack of security of tenure.
He said uncertainty is an enemy of decision-making as leaders are worried and hence do not know who is next on the chopping block.
Prof Baregu said the Constitution Review Commission (CRC) under retired Judge Joseph Warioba proposed for reduction of presidential appointing powers and that hiring and firing of top officials should be done by specific institutions.
The former president of the Tanganyika Law Society (TLS), Francis Stolla, said that civil servants had a right to be heard and must be given a chance to defend themselves before being dismissed.
“There are some civil servants who have known about their suspensions through the media, it is unlawful because they deserve to be officially notified by a written letter”, said Stolla, insisting that law must be observed to ensure good governance.
On his part, the former Controller and Auditor General (CAG), Ludovick Utouh, supported President's Magufuli's ongoing purge of government officials, saying the concept was good as long as there is justification for every suspension or dismissal.
“I think it is being done scientifically and the leaders have all the facts before taking action ... there is no need for civil servants to panic if they are clean and execute their duties perfectly,” he noted.