As the son of James Taylor and Carly Simon, Ben Taylor was born into a musical world with much promise. His older sister, Sally Taylor, found her niche in the industry early on, but the younger Taylor never sought to follow the footsteps of his famous parents. His childhood might have been surrounded by recording sessions and tours across the world, but initially it wasn't in Taylor's mind to become a singer.

Taylor was raised in Manhattan and spent time in various private schools before leaving them behind in ninth grade. By now, he'd taught himself how to play guitar, but traveling the world and connecting with nature was Taylor's passion. Trips across Asia and Europe opened Taylor's eyes to Earth's greatest features, leaving him to think he'd might like to be a gardener or a farmer. Still, he had his hand in a few musical makings. Before he turned 20, Taylor's cover of the Beatles' "I Will" landed on the soundtrack to a domestic comedy featuring Paul Reiser, Bye Bye, Love. He eventually recorded an album for Epic's label The Work Group, but it folded shortly after Taylor inked his contract. He was left feeling disenchanted, unaware what he was supposed to do with his life because he still wasn't convinced music was for him. Taylor would soon come to realize that music was the only career he'd understand.