Putin declared a week ago that Russia had the right to invade Ukraine to protect Russian citizens, and his parliament has voted to change the law to make it easier to annex territory inhabited by Russian speakers.
Speaking by telephone to Merkel and British Prime Minister David Cameron, Putin said steps taken by authorities in Crimea were "based on international law and aimed at guaranteeing the legitimate interests of the peninsula's population," the Kremlin said.
A German government statement, however, said the referendum was illegal. "Holding it violates the Ukrainian constitution and international law."
Merkel also regretted the lack of progress on forming an "international contact group" to seek a political solution to the Ukraine crisis and said this should be done urgently.
On Thursday, Merkel said if a contact group was not formed in the coming days and no progress was made in negotiations with Russia, the European Union could hit Russia with sanctions such as travel restrictions and asset freezes.
Merkel, whose country is heavily dependent on Russia oil and gas, has so far been more cautious than some other nations, urging Western partners to give Putin more time before punishing Moscow with tough economic sanctions.
This stance reflects German fears of the geopolitical consequences of an isolated Russia as much as it does concern about its business interests and energy ties.
In a round of telephone diplomacy on Sunday, the German chancellor also spoke with Turkish Prime Minister Tayyip Erdogan, agreeing that Ukraine's sovereignty must be preserved, and Chinese President Xi Jinping, stressing the need to resolve the crisis through dialog.
In the latest armed action, Russians took over a Ukrainian border post on the western edge of Crimea at around 6 a.m. (0400) GMT, trapping about 15 personnel inside, a border guard spokesman said.
The spokesman, Oleh Slobodyan, said Russian forces now controlled 11 border guard posts across Crimea, a former Russian territory that is home to Russia's Black Sea fleet and has an ethnic Russian majority.
In Sevastopol, several hundred people held a meeting demanding that Crimea become part of Russia, chanting: "Moscow is our capital".
Across town at a monument to Ukrainian poet Taras Shevchenko, violence flared at a meeting to commemorate the 200th anniversary of his birth, when pro-Russian activists and Cossacks attacked a small group of Ukrainians guarding the event and the police had to intervene.
Footage from the event showed a group of men violently kicking one of the Ukrainians as he lay on the ground and a Cossack repeatedly hit him with a long black leather whip.
In Simferopol, Crimea's main city, pro- and anti-Russian groups held rival rallies.
Several hundred opponents of Russian-backed plans for Crimea to secede gathered, carrying blue and yellow balloons the colour of the Ukrainian flag. The crowd sang the national anthem, twice, and an Orthodox Priest led prayers and a hymn.
Vladimir Kirichenko, 58, an engineer, opposed the regional parliament's plans for a vote this month on Crimea joining Russia. "I don't call this a referendum. It asks two practically identical questions: Are you for the secession of Ukraine or are you for the secession of Ukraine? So why would I go and vote?"