Let's say you lend GE $10,000 and collect your 5 percent rate of return in the first year.
At the beginning of the second year, GE starts a new project, requiring a new bond offering. Only now, prevailing interest rates have risen, forcing it to offer 6 percent over the forthcoming decade.
Well, if your neighbor can loan GE $10,000 and receive 6 percent instead of the 5 percent you are receiving, you can see why he surely wouldn't spend $10,000 to buy your bond that only pays 5 percent. A year later your bond might be worth only $9,500, or even less, as a result of a rise in interest rates.
Read MoreNo succession plan? Kiss your clients good-bye
This is why rising interest rates tend to push down the value of bonds on the open market. The inverse is also true, though. When the whole world was gripped with fear in 2008 and 2009, the proverbial "flight to safety," when everyone wanted to own the only presumably default-free investment—U.S. Treasurys—the demand for Treasurys rose, forcing the yields down.
Then the Federal Reserve delivered on its promise to maintain lower rates, effectively and unprecedentedly, by "printing money" and buying U.S. debt instruments, creating artificial demand to keep rates down.