The government says he and the business secretary have “agreed to part ways with mutual consent”.