"No actual or established power has the right to deprive peoples of the full exercise of their sovereignty. Whenever they do so, we see the rise of new forms of colonialism which seriously prejudice the possibility of peace and justice," he said.
"The new colonialism takes on different faces. At times it appears as the anonymous influence of mammon: corporations, loan agencies, certain 'free trade' treaties, and the imposition of measures of 'austerity' which always tighten the belt of workers and the poor," he said.
Last week, Francis called on European authorities to keep human dignity at the center of debate for a solution to the economic crisis in Greece..
He defended labor unions and praised poor people who had formed cooperatives to create jobs where previously "there were only crumbs of an idolatrous economy".
Read MorePope calls for cut in carbon emissions
In one of the sections on colonialism, he said: "I say this to you with regret: many grave sins were committed against the native peoples of America in the name of God."
He added: "I humbly ask forgiveness, not only for the offences of the Church herself, but also for crimes committed against the native peoples during the so-called conquest of America.
"There was sin and an abundant amount of it."
The audience gave Francis a standing ovation when he put on a yellow miner's hat that was given to him at the end of his speech.
The pope made his speech at the end of his first full day in Bolivia, where he arrived on Wednesday. On Thursday morning he said a Mass for hundreds of thousands of people and said that everyone had a moral duty to help the poor, and that those with means could not wish they would just "go away."
Francis praised Bolivia's social reforms to spread wealth under Morales. On Friday, he will walk into Bolivia's notoriously violent Palmasola prison.
The pope looked bemused on Wednesday night when Morales handed him one of the more unusual gifts he has received: a sculpted wooden hammer and sickle - the symbol of communism - with a figure of a crucified Christ resting on the hammer.
Francis leaves on Friday for Paraguay, the last stop on his "homecoming" trip.