He admitted Labour had suffered "a painful defeat" in Thursday's polls but added: "I will not waver. I will not walk away. I will get on with the job."
And he unveiled Glenys Kinnock as Europe minister in a surprise move.
Two more cabinet ministers - Geoff Hoon and John Hutton - have stepped down but neither backed a challenge to the PM.
Caroline Flint has quit as Europe minister, with Glenys Kinnock, wife of former Labour leader Neil Kinnock, who recently stood down as an MEP as her successor - she will be appointed to the House of Lords to be able to take the job.
Ms Flint is understood to have quit after Mr Brown failed to promote her to a full cabinet job.
In her resignation letter she launches a stinging attack on Mr Brown for allegedly treating women ministers "like female window dressing" and running a "two tier cabinet".
She said Mr Brown had "strained every sinew" of her loyalty to the government.
There has been speculation all week that Ms Flint was about to quit, yet she had taken to the airwaves to defend Mr Brown and declare her loyalty to him.
http://news.bbc.co.uk/2/hi/uk_news/politics/8085783.stm
http://news.bbc.co.uk/2/hi/uk_news/politics/8079205.stm
--------------------------------------------------------------------------------------------------------------------------------------------------
Labor party is very unpopular, they are breaking up, and resignation, finger pointing and accusation are going on.