"[With] German yields at such low levels I would not be rushing to put the personal O'Neill family trust in there, that is for sure," O'Neill told CNBC, in his last interview as chairman of Goldman Sach's asset management division.
In his interview with CNBC, O'Neill suggested the recent record lows for European debt yields could be due to increased demand from Japanese investors as they open up to global investment plays.
"For Japanese investors, the incentive to buy many other assets, including foreign bonds, is rising sharply. I think some of the strength of the so-called core markets like France in the past few weeks is probably greatly due to the shift in Japan's policy," O'Neill said on Tuesday.
O'Neill said the "change in the mind set" of Japanese investors reflected recent Japanese monetary policy moves. At the start of April, the Japanese central bank launched a program of aggressive quantitative easing which has helped send the yen to record lows.
O'Neill added that the effects of changing Japanese investor habits could be felt in other asset classes, as well as bonds.
"If you look at equity markets and the real estate sector, it shows some shift is taking place and in fact the gold price might have something to do with Japan," O'Neill said.
One of the key goals of the Bank of Japan's new program is an inflation rate of 2 percent, after being plagued by two decades of deflation. O'Neill said that naysayers of Japan's ambitious target were failing to appreciate the central bank's determination.
"Under Governor Haruhiko Kuroda there has been a huge shift in how the Bank of Japan is thinking. It may well fail, but it is trying to meet a higher inflation target and that is the clarity and power of the message, and why it is so important for people to focus on it," he said.
O'Neill was also optimistic regarding Europe, which he said would continue to face crises which would prove manageable "no matter how bad it seems."
"For Europe, crises come and go and we will get out of whatever crisis it is, and then down the road there will be another crisis, probably from some very unexpected and unpredictable spot," he said.
O'Neill was sanguine as he reflected on his time at Goldman Sachs, which he said had "allowed me to be me." He was cagey about future career plans, but dismissed working for another investment bank, saying that anywhere else would be "downhill" after Goldman.
—By CNBC's Shai Ahmed; Follow her on Twitter @shaicnbc