Your question is a classic case of theory vs. reality.
Certainly, in theory, it makes all kinds of sense to borrow money at under 4% for 30 years, when the return on equities over just about any similar period would be close to 10-11%. I do not believe there would be any 20-30 year periods over the last century in which you would not have come out ahead with this strategy.
In reality, this strategy is full of problems. First and foremost, it assumes that you will have the discipline and patience to not only hang in, but to keep investing during the market drops...even the big ones where it gets really, really scary. Bailing out at the bottom, and being too scared to get back in is probably the number one mistake made by individual investors.
Second, what would happen in at the same time of a market crash you lost a job and did not have sufficient income to pay the mortgage. Then you would be faced with the option of having to sell at a bottom, or losing your house.
Third, you have to decide just how valuable the peace of mind of owning your home free and clear is, versus the potential returns (and losses) that you could get in the market.