In 1509 a charter of "the Lands of Duchra-Estir" was granted by King James IV to Dormand Makersane, who was obliged to build a dwelling house on the land, probably for use as a royal hunting lodge. Early title deeds record an obligation to provide the crown with 70 armed men as and when required and to guard the nearby ford over the Duchray Water.
MacGibbon and Ross' The castellated and Domestic Architecture of Scotland tells how Duchray came into the possession of the Graham family in 1569 with whom it remained until the 1940s. William Graham became first of Duchray and was probably the builder of the castle.
The historian and novelist Nigel Tranter wrote that the castle might have been built on the foundations of an earlier building and that it's massive walls would be highly necessary against the depredations of the neighbouring warlike clan of MacGregor, seldom friends of the Grahams. Nevertheless, there is a story, famously related by Sir Walter Scott, that Rob Roy MacGregor himself was in the Castle when a party of Redcoats arrived to apprehend him. One of the two Graham sisters in residence at the time engaged the officers in talk while the other helped Rob Roy make his escape.
In the 1600s the English set fire to the castle, but suceeded in damaging the roof and timbers, not the walls. Duchray Castle was the meeting point for the Royalist forces raised by by the Earl of Glencairn, who routed Cromwellian English force under General Monk in the Pass of Aberfoyle in 1653, John Graham of Duchray continued to support the King's cause and suffered greatly as a result. Tranter relates that the castle was burned again after the Jacobite rising in 1745 and was certainly recorded as "in ruins" in 1815.
The Castle appears to have be rebuilt in around 1825. The Gothic revival windows and crenellations date from that period, but subsequent alterations through the course of the 19th century and another fire in the mid 20th century have all left their mark on the historic structure.