Scott was raised in the Ozark hills of Missouri. Before leaving Missouri around 1936, he was a teacher in a one-room schoolhouse. After moving to Calfornia, he was a supervisor in the dairy business, an insurance salesman, and then later retired in 1998 after 20 years as a real estate agent. During World War II, he worked for the government as a civilian on the Alaskan-Canadian Highway. A veteran tournament player, who helped originate racquetball in San Diego, he is the recipient of ten gold, bronze and silver U.S. Senior Olympic medals.