James Scouler was born in Edinburgh, the son of an organ builder and music shop owner. He settled in London and studied at the St Martin's Lane Academy. He exhibited at the Society of Artists between 1761 and 1787 and at the Royal Academy between 1780 and 1787. His work resembles that of John Smart, whose initials he shared.