Born in North Adams, Massachusetts, Darby was a self-taught portrait painter, who nevertheless received some instruction from itinerant painters. He may have painted in oils as early as 1842.

In 1847 Darby became a teacher at the South Carolina Female College in Barhamville, returning North soon thereafter, to live in New York City and Brooklyn from 1853 to 1860. During that time, he exhibited portraits at the annual exhibitions of the National Academy of Design. In 1859-60 he showed paintings of John C. Calhoun and Clay.