In the fall of 1957, Olivia McAlister is living in Opelika, Mississippi, caring for her two girls, June and Grace, and her husband, Holly. She dreams of living a much larger life - seeing the world and returning to her wartime job at a landing boat factory in New Orleans. As she watches over the birds in her yard, Olivia feels like an "accidental" - a migratory bird blown off course.
When Olivia becomes pregnant again, she makes a fateful decision, compelling Grace, June, and Holly to cope in different ways. While their father digs up the backyard to build a bomb shelter, desperate to protect his family, Olivia's spinster sister tries to take them all under her wing. But the impact of Olivia's decision reverberates throughout Grace's and June's lives. Grace, caught up in an unconventional love affair, becomes one of the "girls who went away" to have a baby in secret. June, guilt-ridden for her part in exposing Grace's pregnancy, eventually makes an unhappy marriage. Meanwhile Ed Mae Johnson, an African-American care worker in a New Orleans orphanage, is drastically impacted by Grace's choices.
As the years go by, their lives intersect in ways that reflect the unpredictable nature of bird flight that lands in accidental locations - and the consolations of imperfect return.