Born in Vienna, he studied at the Conservatoire from the age of 16 with Fuchs. He composed his first major work, the opera Sarema, in 1895, and it was produced two years later. His progressive late-Romanticism aligned him with Schoenberg, who married his sister. Mahler was another like-minded musician, and he premiered Zemlinsky's opera Es War Einmal at the court opera in 1900. At this time Zemlinsky's own conducting career began to take off; initially as conductor of the Karlstheater. Subsequent appointments at the Theater an der Wien and the Volksoper followed, allowing him to promote both his own works and those of composers such as Korngold.
Zemlinsky taught and conducted in Prague from 1911, and Berlin from 1927. In 1918 he dropped the aristocratic "von" from his name, in sympathy with prevalent republican attitudes. The rise of the Nazis forced him, as a Jew, to flee Berlin in 1933. Relocation to Vienna was not to prove a permanent solution, and in 1938 he fled to America.
Many composers of the time held Zemlinsky in high regard, Schoenberg especially. Their early works bear comparison; adventurous tonality, frequently imposing stretches of Wagnerian Romanticism and comprehensive use of a large orchestra. Berg was to quote from Zemlinsky's 1923 Lyric Symphony, probably his most performed work, in his Lyric Suite.