Gilliam was a star athlete for the Monarchs and then was an All-Conference quarterback for Alcorn A&M College (now Alcorn State University), where he also played baseball. Gilliam was also an assistant football coach at Magnolia High School and head coach for Nichols High School in Biloxi.

As a young boy in Moss Point, Gilliam was one of the early members of Boy Scout Troop 400, which remained active as a troop until the mid-1970s.