Senator Brown secrets about his childhood of abuse were revealed in his memoir, titled Against All Odds. He explains that the title not only describes his political upset of the liberal icon but also his life story. It talked about how he had to resort to stealing to feed his younger sister and himself.

There was also a section about enduring beatings made by two stepfathers. He also had to fight off two separate sexual predators. Not even his wife nor his parents knew about the sexual abuses he experienced in two separate occasions.

He told USA Today that everything that happened in the past made him a strong person today. His past experiences made him the protector of the family. He added that he learned that it is necessary to fight in case you wand to achieve.

Senator Brown has no regrets and shows no resentment toward his mother, who was an alcoholic, and his absentee father who was gone when he was still a toddler. He is now trying to build a positive relationship with his parents.