Abe accepted Imamura's resignation.
"I, as the prime minister, bear the responsibility for appointing him".
"I have to assume that we'll see a cabinet reshuffle over the summer - perhaps bigger than previous ones - in order to reassert more control heading into the a year ago of Abe's term", Harris said.
A Japanese cabinet minister is set to resign after saying it was better that the northeast of the country was hit by the 2011 quake and Fukushima nuclear disaster, and not Tokyo.
Mr Putin and Mr Abe told reporters after the talks at the Kremlin that a group of Japanese officials and businessmen would travel to the Kurils next month to examine opportunities for Japanese investment. However, the presence of more than 120,000 people displaced from their homes, including almost 80,000 Fukushima residents, signals that the reconstruction of people's lives shattered by the disaster clearly remains a work in progress.
"If it had been close to the capital zone, there would have been enormous damage", said Imamura, who later retracted the remarks. Abe, who later attended the same party, immediately offered an apology himself, stating that "there has just been an extremely inappropriate remark that hurts the sentiments of people in Tohoku".
In the wake of Imamura's resignation, Abe said reconstruction of the areas hit by the 2011 disasters will be the top priority of his administration.
Former deputy environment minister from Fukushima and liberal Democratic Party's Masayoshi Yoshino has been appointed as the new reconstruction minister.
Shortly after Imamura made the remarks on Tuesday, the prime minister made a brief speech at the same fundraising party.
It was not the first time Imamura has courted controversy.
In reply to a question about government responses to those who have voluntarily evacuated areas hit by the Fukushima nuclear disaster out of concerns about radiation, the new reconstruction minister said, "We'd like to extend further assistance if aid provided so far is insufficient". In recent months, a number of gaffes and scandals have hit his Cabinet, although approval ratings have remained high at around 50 percent.
A vice economy minister resigned last week after a magazine reported on an extramarital affair. "There have been criticisms that we have been too lax, and I take them seriously".