Defectors included Shin Dong-hyuk, who gave harrowing accounts of his life and escape from a prison camp. As a 13-year-old, he informed a prison guard of a plot by his mother and brother to escape and both were executed, according to a book on his life called "Escape from Camp 14".
Kirby said that the crimes the team had catalogued were reminiscent of those committed by Nazis during World War Two.
"Some of them are strikingly similar," he told Reuters.
(Read more: North Korea executes supreme leader's uncle)
"Testimony was given ... in relation to the political prison camps of large numbers of people who were malnourished, who were effectively starved to death and then had to be disposed of in pots, burned and then buried ... It was the duty of other prisoners in the camps to dispose of them," he said.
The number of North Korean officials potentially guilty of the worst crimes, would be "running into the hundreds", he said.
The independent investigators' report cited crimes including murder, torture, rape, abductions, starvation and executions.
"The gravity, scale and nature of these violations reveal a state that does not have any parallel in the contemporary world," it said.
North Korea's diplomatic mission in Geneva dismissed the findings. "We will continue to strongly respond to the end to any attempt of regime-change and pressure under the pretext of 'human rights protection'," it said.
(Read more: Rodman leaves for North Korea to play basketball on leader's birthday)
The two-page North Korean statement, in English, said the report was an "instrument of a political plot aimed at sabotaging the socialist system" and defaming the country.
Violations listed in the document and forwarded to Pyongyang for comment several weeks ago, "do not exist in our country".
The investigators said abuses were mainly perpetrated by officials in structures that ultimately reported to Kim - state security, the Ministry of People's Security, the army, the judiciary and Workers' Party of Korea.
"It is open to inference that the officials are, in some instances, acting under your personal control," Kirby wrote in the three-page letter to Kim published as part of the report.
The team recommended targeted U.N. sanctions against civil officials and military commanders suspected of the worst crimes. It did not reveal any names, but said it had compiled a database of suspects from evidence and testimony.
(Read more: North Korea frees US Korean War veteran after 7 weeks)
Pyongyang has used food as "a means of control over the population" and "deliberate starvation" to punish political and ordinary prisoners, according to the team of 12 investigators.
Pervasive state surveillance quashed all dissent, it said.
North Korea's extermination of political prisoners over the past five decades might amount to genocide, the report added, although the legal definition of genocide normally refers to the killing of large parts of a national, ethnic or religious group.
Kirby warned China's charge d'affaires in Geneva, Wu Haitao, in a Dec 16 letter that the forced repatriations of North Korean migrants and defectors might amount to "the aiding and abetting (of) crimes against humanity", the said.
Wu, in a reply also published in the report, said the fact that some of the North Korean migrants regularly managed to get back into China after their return showed that the allegations of torture were not true.
Human Rights Watch said it hoped the report would open the U.N. Security Council's eyes to the scale of atrocities.
"By focusing only on the nuclear threat in North Korea, the Security Council is overlooking the crimes of North Korean leaders who have overseen a brutal system of gulags, public executions, disappearances, and mass starvation," said executive director Kenneth Roth.