James Duff, 5th Earl of Fife (1814-1879)/descendants