On the surface, it might seem that diversification should be a matter of preference. Those with a high risk tolerance might rationally hold undiversified portfolios in the hopes of getting outsized returns, while those that want lower “swings” hold a more diversified portfolio.

This is the case, for instance, in dice. Suppose you are betting on the next number to come up on a single dice. The odds of being right about any one number are 1/6. You could bet on three numbers and you would have a 3 in 6 (50%) chance of being right, but this bet would cost three times as much as a bet on one number, so in the end it does not really matter whether you bet on one number or three. Whether you are diversified or undiversified, if you play the game enough times your returns will be the same.

But investing is different from playing dice. In investing, diversifying actually creates value. Diversifying your investments allows you to earn greater returns without taking on more risk, or equivalently to take less risk without sacrificing any returns.

The second of these – how you can reduce the risk of a portfolio with diversification – is pretty clear. Imagine you are given two investments to choose from. Both have an expected annual return of 8%, but both also have a 1 in 5 chance of losing 50% over the next year. If you buy either investment, you will expect to make 8% but have a 20% chance of losing 50%. But what if you buy equal amounts of both investments? Your expected return is still 8%, but now your chance of losing 50% is only 1 in 25 (4%), because it would take both companies having unusually bad years for you to have a worst-case scenario (this assumes the investments are unrelated, that is a bad year for one is not more likely than normal to be a bad year for the other). So your average return is the same, but your risk is smaller. A win-win.

Let’s look at another example to see why diversification also can increase returns while keeping risk the same. Suppose that a portfolio invested entirely in stocks runs the risk of losing up to 60% of its value (this approximates the historical peak-to-bottom loss of the market in a really bad market). Not many people are able to withstand a 60% loss. What if you are only able to accept the risk of a 30% loss because of your income needs? One solution is to hold 50% cash and 50% stocks, so your overall portfolio will never fall by more than 30%. Nowadays, however, the return on cash is next to nothing, so you would reduce your total returns by 50% as well. If you expected to earn 8% on average with your all-stock portfolio, you will only earn 4% now.

But if you also have the option to invest in another asset – perhaps called “bonds” – that tends to do well in periods when stocks are going to suffer, then your outlook completely changes. Suppose bonds will only return 3% a year on average, but they will return 10% in years in which stocks lose 60%. Now a 50% bonds 50% cash portfolio can be expected to lose 25% in a really bad year, while still making 5.5% on average. But because bonds are diversifying your risk, this actually lets you put even more money into stocks, so you can increase your return even more. Paradoxically, diversifying lets you increase the amount of your money in high-risk assets while accepting the same level of risk as before.

This ability to combine uncorrelated asset classes to smooth returns is extremely important when it comes to asset allocation (more on this in the next chapter). During many periods of recent history, one asset class has done particularly well while another has faltered. For instance, in the 1970s, commodities had a great bull market while stocks performed poorly. In the ugly aftermath of the dot-com bubble, it was real estate that had great performance. And in 2008, Treasury bonds went on a tear when the equity markets were falling apart. So a portfolio of stocks, bonds, real estate, and commodities has lower risk than a portfolio of equities alone, and this lower level of risk, somewhat paradoxically, allows you to keep more money in risky assets that could fall in value, thereby earning greater returns.

This remarkable property is why diversification is a “free lunch.” It allows you to avoid the usual trade off between risk and return. If you diversify an undiversified portfolio you can get higher returns without increasing the chance of losing money, or you can reduce the chance of losing money without hurting your long-term returns. Nowhere else in finance is this the case. For another explanation of diversification, see this article.