Willis was born in Idar-Oberstein, Germany, the son of a Kassel-born German mother, Marlene, who worked in a bank, and David Willis, an American soldier. Willis was the oldest of four children (his siblings are Florence, David, and Robert). After being discharged from the military in 1957, Willis' father took his family back to Penns Grove, New Jersey.