Why were people who did not invest in the stock market impoverished by its fall?

Because most people did. They might not have been affected, but their employers, grocers, neighbors, and mostly everyone around them were. So because their employers were crippled, so were their pay-checks. The grocer has no money to buy products, so they couldn't get food. Couldn't even ask for help from friends, or neighbors. 

There’s no picture, but i’ll give you a few of expeditions. lewis and clark began in 1804, louisiana purchase 1802, james town 1607