The office was set up to facilitate communications between the governments of President Jae-in and North Korean leader Jong Un.
Moon's administration had touted the office as a major feat resulting from his own summit with Kim a year ago despite US concerns about possible loosening of sanctions.
"The North's side pulled out after conveying to us that they are doing so on the instructions from a higher level, during a liaison officials' contact this morning", South Korea's Vice Unification Minister Chun Hae-sung told a briefing.
When it opened Seoul's unification ministry said it would become a "round-the-clock consultation and communication channel" for advancing inter-Korean relations, improving ties between the U.S. and the North, and easing military tensions.
North Korea abruptly withdrew its staff from an inter-Korean liaison office in the North on Friday, Seoul officials said.
Moon has long backed engagement with the North to bring it to the negotiating table, and has been pushing the carrot of inter-Korean development projects, among them an industrial zone also in Kaesong and cross-border tourism for Southerners.
The liaison office opened last September in the city of Kaesong just north of the border between the two Koreas to foster closer bilateral ties. It didn't say whether the withdrawal would be temporary or permanent.
North Korea said it was quitting the joint liaison office set up in September in the border city of Kaesong after a historic summit between leader Kim Jong Un and South Korea's President Moon Jae-in early previous year.
WATCH | North Korea vs.
Moon's office said presidential national security adviser Chung Eui-yong convened an emergency meeting of the National Security Council to discuss the North Korean withdrawal.
The North Korean action came a week after its vice foreign minister threatened to pull out of nuclear negotiations with the United States, citing a lack of US steps to match disarmament measures it took previous year.
U.S. -North Korean engagement has appeared to be in limbo since the February 27-28 summit, despite U.S. Secretary of State Mike Pompeo saying on March 4 he was hopeful he could send a team to North Korea "in the next couple of weeks".
The town is where the Koreas' now-stalled jointly run factory complex was located. North Korean state media have recently demanded that South Korea distance itself from the USA and resume joint economic projects that have been held back by the US -led sanctions against the North. It combined South Korean capital and technology with cheap North Korean labor. -North Korea summit in Vietnam collapsed due to disputes over US -led sanctions on the nuclear-armed North.
The Treasury Department said it was blocking all potential U.S. interests of the Dalian Haibo International Freight Co.