His remarks come just two days after Prime Minister Theresa May announced that Britain would not remain a member of the common market once it leaves the EU. The EU's single market allows its members to trade goods and services freely, but it also involves free movement of people - something that Theresa May does not want.
The mayor of London has told CNBC he has met business leaders and speaking to executives and entrepreneurs, across London and the feedback is that "the access to a single market (and) the continued ability to attract talent is important for jobs, growth and prosperity."
"One of the things I am doing is not just lobbying the British government but lobbying businesses to press upon their governments the fact that a hard Brexit would be bad for London and for Europe too," Khan said.
"The concern is some of these businesses may decide to leave London and go not just maybe Paris, Madrid or Berlin, but leave Europe altogether, and go to Hong Kong, Singapore," Khan added.
Prime Minister Theresa May is also in Davos this Thursday to reassure investors about her Brexit plans.