Another theory I have on genealogical brick walls is that they are the product of dysfunctional families. I define dysfunctional not in the modern psychological sense, but in a genealogical sense that it was a family that differed from the norm: mom and dad with kids who grow up, get married, and mom and dad grow old and die. Anything that disrupts this norm such as a single parent, being orphaned, or a family life that wasn't great results in a break from the larger family and may lead to two things that impedes genealogy: (i) your line loses touch with the other family members so there is no oral history passed down; (ii) the larger family loses track of this line and it is not included in compiled genealogies.
Tolstoy wrote that every unhappy family was unhappy in its own way. If you consider all the families from which we descend it's only sensible that they span the Bell Curve of happy to unhappy. Some of the couples married for love, most didn't. Some of the families were happy and some weren't.
Here's two examples and they have affected my research, I feel, tremendously. My 100% Yankee great-grandfather Burt Hale Pinkham was born in 1881 in Dover, New Hampshire. His family, and the families of both his parents had real troubles. His father, George Hale Pinkham 1843-1888, served in the Civil War for its entirety. Whether the war had an impact or not, he clearly became delusional and violent. He was committed to the State Asylum where he died. I've always suspected [and clearly I cannot prove or disprove these theories] that his brothers didn't want anything to do with him. He had one brother who became a prominent citizen, serving as Free-Will Baptist minister and a representative to the state legislature of New Hampshire. Despite that, and having no children, when Charles Luther Pinkham died he mentioned none of nieces and nephews. George was a shoemaker and when he died, he left a widow with four children. She had no education and took in washing for a living. So this family was in the lower economic levels. My great-grandfather only kept connections with his own sisters and somewhat with his mother's family. However, the Pinkham family connections had been forever severed.
George's wife was Olive Ann Hurd 1851-1934. Her father Benjamin Wingate Hurd abandoned their family in 1860. Ostensibly he went west to work and then served in the Civil War in the cavalry in Kansas. Benjamin did serve in the Civil War. He just never returned to Dover, N.H. and had another family in Kansas City. This left his wife Abiah Russell Learned 1830-1926 in poverty as well. Leaving a woman in the 19th century was not only immoral, it was criminal. She could not own property for the most part. She probably had no education. And occupations for women were scarce. She was at the mercy of her own family or her in-laws for help. And certainly in the 1860 U.S. census, Abiah is living with her mother-in-law and next door to her brother-in-law. So in two successive generations Abiah and her daughter Olive had to raise their families on their own. I can't tell you how much respect and awe I have for those two women. It's probably a large part of why I love trying to figure out female genealogy so much. Their stories and places in history really need to be told.
Maybe I've read too much Dickens and Austin, but you get the feeling that the branches of the family that had these sort of problems, were pushed to the side. You can call it the Poor Relatives Syndrome. In any case, as I've stated the families split and forget each other. The continuity of tradition is broken. And the relatives in poverty don't have time or resources to worry about their own ancestors or saving records from them to pass on. Life was the struggle. Genealogy should never rank high when you need to put food on the table.
Remarkably you can actually benefit if your ancestors were truly, truly poor. Pauper records are great to use and have helped me solve some problems for others. I think my ancestors were those just above that level--subsistence poor, if you will. In any case, the loss of a parent or any sort of major shake-up in a family probably means trouble for the genealogist 150 or more years later.