The prison was built in 1861 by the French colonists to jail those considered especially dangerous to the colonial government. Many of the high-ranking leaders of Vietnam were detained here. It is ranked a special historical relic of national importance by the government of Vietnam. The most famous site in this prison are the "tiger cages" (vi:"chuồng cọp"). The French tiger cages cover an area of 5.475 m2, within which each cell occupies 1.408 m2, solariums occupy 1.873 m2, and other spaces occupy 2.194 m2. The prison includes 120 cells. The prison was closed after the end of the Vietnam War and opened for visitors soon after.

In 1861, the French colonial government established a prison on the island to house prisoners who had committed especially severe crimes. After the turn of the century, the prison held an increasingly larger population of political prisoners. In 1954, it was turned over to the South Vietnamese government, who continued to use it for the same purpose. Notable prisoners held at Côn Sơn included Phan Châu Trinh from 1908-1911, and Tôn Đừc Thắng, Phạm Văn Đồng and Lê Đức Thọ in the 1930s. Võ Thị Sáu was executed at the prison in 1952 (though she was imprisoned at the police post outside of the prison).Not far from the prison is Hàng Dương Cemetery, where some of the prisoners who died between 1941 and 1975 were buried.

During the Vietnam War, prisoners who had been held at the prison in the 1960s and 70s were abused and tortured. In July 1970, two U.S. Congressional representatives, Augustus Hawkins and William Anderson, visited the prison. They were accompanied by Tom Harkin (then an aide), translator Don Luce, and USAID Office of Public Safety Director Frank Walton.

When the delegation arrived at the prison, they departed from the planned tour, guided by a map drawn by a former detainee. The map led to the door of a building, which was opened from the inside by a guard when he heard the people outside the door talking. Inside they found prisoners were being shackled within cramped "tiger cages". Prisoners began crying out for water when the delegation walked in. They had sores and bruises, and some were mutilated. Harkin took photos of the scene. The photos were published in Life magazine on July 17, 1970.

According to VietKings (