The fort was built by the Spanish, ceded to the British by treaty, then back to Spain by treaty at the end of the Revolutionary War, then to the US after the Civil War. It was besieged and attacked several times but never fell.