The barque Scindian brought the first convicts to Western Australia in 1850, and with them a number of Enrolled Pensioners. Transportation ceased in 1868, after 9,721 convicts had arrived in the Colony, the last convict ship being the Hougoumont.
A very large area was set aside for the Convict Depot, which included various dwellings as well as the Prison. See:
('Discipline') Warders Cottages
Supervising Warders (sappers) cottages
Government House (Henry Wray's house) aka Marmion Cottage
The Knowle (Edmund Henderson's house)
North Fremantle depot
This part of a c. 1885 map shows the boundaries as: Hampton Rd on the right-hand edge; Alma Rd at the bottom; South Terrace and Henderson St to the left; and Holdsworth and Knutsford Streets (not named) at the top. The number 29 designates The Knowle (still in situ in the grounds of Fremantle Hospital), the grand residence of Henderson, the Comptroller-General. No. 30 (now the site of part of Fremantle Hospital) is the residence built originally for Henry Wray, chief of the Royal Engineers, which was later called 'Government House' (tho the Governor rarely stayed there), and finally Marmion Cottage, from one of its last residents, W.E. Marmion. No. 28 is the Barracks, the quarters of the Enrolled Pensioner Force. On that site now is the Stan Reilly Centre, which is soon to be demolished for a carpark. On the left, cnr Henderson St and South Terrace are a pair of semi-detached houses, the residents of 'instructing warders' (sappers who instructed and supervised the work of convicts). Running up Henderson St are the three blocks of Warders Cottages which are still standing, have just been renovated, and are now being sold. They were built to house the 'discipline warders', that is, the gaolers, turnkeys. Finally, on the corner of Henderson and Queen (then Doonan) Streets are the main quarters of the sappers (later Royal Engineers), from the left, the single men's quarters, the offices (top), and married quarters (the longest building). That site is now the position of the (former) police station and courthouse.
This photo of the prison was taken by Stephen Stout in about 1865, not long after the building was completed in 1859.
Another version of Stout's photo, from another source, Battye 88282.
Edmund Henderson RE, 1 Feb 1850-63
Henry Wray RE, 26 Feb 1856-58
Edmund Henderson RE, 1 Feb 1858-63
Charles Frankland Newland, 15 Jan 1863-66
George Essex Hampton, acting, May 1866-67
Henry Wakeford, 16 May 1867-72
Willliam Fauntleroy, 1 Mar 1872-79
John Hampton arrived as Governor in 1862. Having been a harsh Comptroller-General of Convicts in Van Diemen's Land it was perhaps understandable that he took more control over prison affairs than previous governors. He was a cruel man, who clashed with Newland, and eventually had him removed, replacing him, nepotistically, with his son George. Both Hamptons were extremely unpopular in the colony until John Hampton left in 1868.
The most notable event of 1850 was the arrival, on June 1st, of the ship Scindian with the first batch of convicts, the date synchronising with the twenty-first anniversary of the foundation of the colony. In addition to the convicts and military guards with their wives and families there arrived by the Scindian, Captain Henderson, R.E., Comptroller-General of convicts, and a large staff of officials, including Dickson [sic], principal overseer, and James Manning, clerk of works. Later a number of tradesmen were brought from South Australia to instruct the convicts in the various trades in which they were to be employed.
As no preparation had been made for the reception of the convicts, the first batches were quartered in premises rented from Captain Daniel Scott, fronting Marine Terrace, Collie Street and Essex Street, extending half-way-up to Essex-lane. The only part now  remaining is used as a warehouse and extends from the Esplanade Hotel to Essex-street.
That temporary prison reverted to Captain Scott when the convicts were transferred to the new prison erected by themselves on the hill. The numerous outbuildings at the rear of the main building were then converted into tenements which, in the course of time, degenerated into mere hovels and what was known as the Old Establishment Yard became the slum quarters of Fremantle in the [18-]sixties and seventies, the tenants being mostly ticket-of-leave men and their consorts, the demimonde always to be found in a seaport town.
”Bond or Free?”
From 1850 to 1868, in which year transportation ceased, 9,721 convicts were landed at Fremantle. In the convict days a bell was rung at 9.50 p.m., and anyone in the street ten minutes after it ceased ringing who could not give the answer ”Free” to the policeman's challenge of ”Bond or Free?” was promptly escorted to the Round House. At the time of writing  that old building seems doomed to demolition. Steps in that direction had been taken when the protestations of those interested in the preservation of relics of a bygone day prevented the intention from being carried out.
As an offset against the influx of the criminal element the convict system was the means of introducing about 2000 persons consisting of military pensioners with their wives and children. A contingent of those time-expired soldiers came out as guards over the prisoners in every convict ship. A large number of them were retained in the enrolled force to guard the convict establishment and others became warders or policemen or entered into other pursuits. Many of them were veterans of the Crimean War and Indian Mutiny, and others were comparatively young men who had been invalided home from India early in their military career. On the whole they were a valuable addition to the population; numbers of them accumulated property, and many of their descendants are to be found occupying prominent positions, both in private and official life.
The Convict Establishment
The erection of the Convict Establishment was commenced in 1851. In that and the following years the Imperial Government carried out by prison labour a number of other works, among them being the Commissariat buildings in Cliff-street, the pensioners' barracks in South Terrace, the warders' quarters in Henderson Street, the Comptroller's residence, known as ”The Knowle” (now part of the hospital), the North Fremantle [first] traffic bridge as originally constructed and numerous other works of public utility, including roads, streets and public buildings. With the exception of the old court-house, which was demolished to make room for pilots' quarters, all those buildings and the North Fremantle bridge are still standing, and are monuments to the stability of the work done by the convicts.
The convict system is referred to frequently in this history, and it may be well to emphasise that the class of men transported to the colony bore no comparison with the desperadoes who were sent out to New South Wales and Tasmania. Western Australia was made a penal settlement at the request of the settlers in a time of stress and in the agreement made with the British authorities it was expressly stipulated that neither criminals of a reckless or dangerous class should be sent out, nor should any female offenders be transported to the colony. The latter condition was strictly carried out, but a portion of the male prisoners proved not up to specification, and it was from those that the chain gangs were formed and recruited. The majority of the convicts proved sincerely desirous of becoming good colonists and succeeded in doing so. Among them were lawyers, doctors, teachers, ex-bankers, journalists, ex-clergymen, civil engineers, architects, master mariners, accountants and others whose presence in the community tended to elevate rather than lower the moral standard of the people. At one time records existed in the prison of particulars of the crimes for which the convicts were tried and convicted in England. Those particulars, sometimes maliciously amplified, of every ex-convict's crime and punishment at times became public property, but on the inauguration of responsible government all those records were burnt. Hitchcock: 33-35.
Bosworth, Michal 2004, Convict Fremantle: A Place of Promise and Punishment, UWAP.
Bridge, Calliope et al. eds, 2017, Prisoners of the Past, Hesperian.
Broomhall, Frank H 1975, The Veterans: A History of the Enrolled Pensioner Force in Western Australia 1850-1880, Hesperian Press.
Campbell, Rob 2017, Henderson & Coy: Royal Engineers & the Convict Establishment Fremantle, WA, 1850-1872, Faculty of Architecture, UWA.
Clarke, Lorraine & Cherie Strickland 2017, Australia's Last Convicts: Reprobates, Rogues and Recidivists, Friends of Battye Library.
Convict Records of Western Australia: A Research Guide, 2016, Friends of Battye Library, rev. ed.; first publ. 1990.
Crawford, Ian, Anne Delroy & Lynne Stevenson 1992, A History of the Commissariat, Fremantle 1851-1991, WA Museum.
Derrincourt, William 1889, Old Convict Days, ed. Louis Becke, T. Fisher Unwin, London; repub. Penguin, 1975.
Edgar, W.J. 2012, 'Lags': A History of the Western Australian Convict Phenomenon, Tammar Publications, Doubleview.
Gertzel C.J. 1949, The Convict System in Western Australia 1850-1870, Hons dissertation, UWA.
Gill, Andrew 2004, Convict Assignment in Western Australia: 'The Parkhurst Apprentices' 1842-1851, Blatellae Books, Maylands; republ. 2016, Hesperian, Carlisle.
Hasluck, Alexandra 1959, Unwilling Emigrants: A Study of the Convict Period in Western Australia, OUP, Melbourne; repub. A&R, Sydney, 1969; repub. FACP, 1991 [incl. bio of convict William Sykes].
Hitchcock, J.K. 1929, The History of Fremantle, The Front Gate of Australia 1829-1929, Fremantle City Council.
James, MS 2016, A Superior Body of Men, Authorhouse. [Adds further biographical information to the history of the military pensioners who were either discharged in the Colony of Western Australia, came as guards on the convict transports, or immigrated freely to the State.]
Kerr, James Semple 1998, Fremantle Prison: A Policy for its Conservation, Dept of Contract and Management Services for the Fremantle Prison Trust Advisory Committee, rev. ed.
Millett, P.R. 2003, A Mild but Firm System of Discipline: British Convicts and their Punishment in Western Australia 1850-1886, PhD dissertation, UWA.
Reece, Bob ed. 1991, Exiles from Erin: Convict Lives in Ireland and Australia, Macmillan.
Reeves, Noeline 1994, The Development of a Literate Society: A Case-Study of Western Australia 1850-1914, PhD dissertation, Murdoch.
Sherriff Jacqui & Anne Brake 2006, Building a Colony: The Convict Legacy, UWAP.
Trinca, Mat 1993, Convicts and the Controlling Space in Western Australia 1850-1870, Hons dissertation, UWA.
Conole, Peter & Diane Oldman 2016, 'Edmund Henderson: Comptroller-General of WA Convicts', Western Ancestor, vol. 13, no. 5 March.
Cullity, Olimpia 2016, 'Reform and punishment: Fremantle Prison, 1850 to 1891', Studies in Western Australian History, 31: 63-79.
Gibbs, Martin 2001, 'The archaeology of the convict system in Western Australia', Australasian Historical Archaeology, 19.
Heseltine, William 2004, 'The escape of the military Fenians from Fremantle Prison: the warders' perspective', Fremantle Studies, 3: 26-45.
Megahey, Norman 2004, 'Adaptation and resistance: the reaction of Fremantle Prison inmates to incarceration', Fremantle Studies, 3: 14-25.
Megahey, Norman 2010, 'A community apart', Fremantle Studies, 6: 29-42. [on Fremantle Prison riot 1988]
Murray, Sandra 2007, 'Escape! Fremantle to Freedom: an exhibition on the Irish Fenian convicts and their bold escape from the Fremantle Prison to America', Fremantle Studies, 5: 74-86.
Taylor, Sandra, 'Who were the convicts? A statistical analysis of the convicts arriving in Western Australia in 1850/51, 1862/62, and 1866/68', part of an MAPrelim dissertation, History, UWA, 1978.
Trinca, Mat 1997, 'Controlling places', Studies in Western Australian History, 17: 13-34.
Winter, Sean 2013, Global, regional and local networks: archaeological investigation of the Western Australian Penal Colony 1850-1875, PhD dissertation, UWA.
Fremantle Prison convict database
Bibliography on the Prison website
Ships that brought convicts
Wikipedia page for Comptroller General of Convicts
Page on convict hiring depots on Diane Oldman's RE site
Garry Gillard | New: 6 October, 2016 | Now: 11 July, 2018