John Donovan Cannon graduated from high school in his hometown of Salmon, Idaho, in 1940. His subsequent studies at the Academy of Dramatic Arts were interrupted as a result of wartime military service from 1942 to 1945. After the war, Cannon returned to New York to complete his training and acting on the stage, both on and off-Broadway, trying his hand at a wide variety of parts, mostly in classical plays. He essayed "Petruchio" in "The Taming of the Shrew" and appeared on Broadway in "Henry IV", "Lysistrata" and "Peer Gynt". Leading roles were few and far between, however.

Once Cannon had found his niche as a frequent guest star on numerous television episodes, his career as a motion picture actor became somewhat desultory, though he had memorable roles in two films: as the road gang convict "Society Red" in Cool Hand Luke (1967); and as the gangster "Calhoun" in Cotton Comes to Harlem (1970). His first TV appearance was as a poker-hustling master sergeant in a 1958 episode of The Phil Silvers Show (1955). His tough screen persona was pretty well established by the mid-60's, though, against character, he portrayed the timid weakling "Lloyd Chandler" (witness to the original crime committed by the "one-armed man") in the final denouement installment of The Fugitive (1963). He was at his scene-stealing best in an episode of The Invaders (1967) as "Peter Kalter", a strangely sympathetic mobster who turns against his own outfit and aids the chief protagonist against the impending alien threat.

In his recurring role as lawman "Harry Briscoe" in the western comedy series Alias Smith and Jones (1971), Cannon was again given the opportunity to deliver some enjoyably caustic one-liners, something at which he excelled. He will arguably be most fondly remembered for his popular portrayal of the perpetually exasperated, choleric, cigar-chewing NYPD Chief of Detectives, "Peter B. Clifford", in McCloud (1970) (1970-77).