This "substantial degree of home bias" also applies to debt securities and other assets, he said, suggesting globalization still has some way to go when it comes to cross-country financial integration.
U.S. stock markets captured slightly less than 40 percent of global capitalization, according to Ursua. In his view, this means that U.S. investors should invest about 60 percent of their portfolios abroad, rather than the current 16 percent.
Read More Goldman Sachs: It's showtime for global growth
However, Hugh Johnson, chairman of Hugh Johnson Associates, believes that while investors should have an international foothold, they should be underweight international stocks.
"The relative performance of [both the developed countries and emerging countries] tells me you want to really have low exposure to international and high exposure to the U.S.," he told CNBC's "Power Lunch."
When it comes to picking international names, he suggests sticking with the developed countries, since emerging markets like Brazil, India, South Africa and Turkey still have significant problems.
With better technology, more established financial markets and fewer rules on moving capital overseas, investors have been able to diversify their portfolios over the last few decades. However, this global diversification trend hasn't played out as expected, according to Goldman's research, with portfolios still lacking cross-country holdings.
Ursea highlighted the case of euro zone banks holding high levels of sovereign bonds as an example.
Read MoreWe need smart regulation: Goldman Sachs
Following the global financial crash of 2008, euro zone banks bought large volumes of debt issued by their home countries, which was classed as risk-free under banking regulations. However, analysts warn that the link between weak governments and weak banks is creating a "doom loop" with the potential for banks and sovereigns to drag one another down in a future crisis.