Segal was born to Jewish parents in Iaşi, Romania, and studied at the Berlin Academy from 1892. He studied with Schmid-Reutte [de] and Hölzel in Munich in 1896, and later studied in Paris and Italy in the early 1900s.
After studying in Paris and Italy, he eventually moved to Berlin in 1904 where he exhibited his work with Die Brucke and Der Blaue Reiter, two leading German expression groups. In 1910 he co-founded the Neue Sezession, a group of artists whose work was rejected by the Berliner Sezession.
On the outbreak of war in 1914 he moved to Ascona, Switzerland with his family, and remained there until 1920. During the war, Switzerland became a refuge for many artists like Segal, and whilst he was there he exhibited some of his work with Arp and Dada in the Cabaret Voltaire in Zurich. In 1920, he returned to Berlin, where he founded his own art school in 1923, Novembergruppe. He was offered a teaching job at the Bauhaus in Dessau, but he declined. Because of his Jewish background he was prevented from exhibiting his work in Germany, and so in 1933 he moved to Palma, Majorca and then London, where he set up another school with his daughter Marianne. He died in London.
Segal was firstly a painter, and his early work was heavily influenced by impressionism and neo-impressionism. From around 1910 he began a more expressionism and dadaism style, and around 1916 found his own modern style. As well as painting, he also produced woodcuts from 1910, many of which were anti-war themed. Segal was also the author of many books, articles, and often gave lectures.