John Robertson was born in 1923 to a Catholic father and Jewish mother, who had met as students in Munich. They were communist intellectuals. Hilda worked for the Komintern in Berlin. They also had a daughter, Brigitte, born in 1926. Hilda had to leave Berlin for political reasons and took the family initially to Amsterdam, where she had a sister, and later to Paris where she worked for the Oevre de Secours aux Enfants and managed an orphanage. In the meantime she had separated from her husband, Friedrich, who subsequently remarried to an Argentinian and had two further children. 

John was sent to Great Britain in 1938 and his mother and sister came later. His mother entered a marriage of convenience with Dale Robertson, the son of the family which hosted them in Highbury. The resultant British passport enabled her continued residence in Great Britain.