Born in China in 1925 of Chinese and German parentage, Oliver in his childhood lived in China, Germany, Mauritius, Iran, and the U.S. He was educated at Harvard, Johns Hopkins University, and the University of Chicago. He was Professor of Political Science at the University of Hawaii from 1963 to 2001. He and his wife, May Lee, have three children and five grandchildren, almost all living in Hawaii.