Building the castle
In true eccentric-rich-guy-style, Bannerman ordered that his private-island munitions warehouse resemble a Scottish castle. He drew up the ornate designs himself. Most of the construction happened without the help of professionals like engineers, architects, and contractors. And of course because he had an enormous weapons stockpile, he had plenty of powder to blow up bits of the island in order to level it for building.
Since the building was visible to both train passengers and tourist boats, he ensured that the side of the structure was emblazoned with the words "Bannerman's Island Arsenal," written in four-foot-high-letters. The castle served as a particularly classy billboard. Upstate New York was a popular tourist destination, so there were many, many passengers heading upstate or sailing along the Hudson. So as eccentric as the castle's vibe is, it was a smart marketing tactic.
And hey, it still works today--I first learned about the castle's existence by looking out of the train window, after all.
A regal warehouse
A weird, folk-art adjacent castle is like catnip to me. This isn't the only American castle (built by some random guy) that I've toured. Like I mentioned, Bannerman's castle was built primarily by day-laborers, without the guidance of the typical experts you might usually bring in. I'm not sure whether this was a cost-saving tactic or something he did to preserve his vision for the place.
If I recall correctly from the tour (it was a couple years ago), the castle is constructed from cement and brick. According to the New York Times, they also "used recycled bedsprings, bamboo spears and bayonets as building materials."[^1] I seem to recall that it also incorporated some real cannon balls, as well. Originally, the castle was complete with a drawbridge, a moat of thistles, and a portcullis.
Unfortunately, the castle itself hasn't exactly stood the test of time, falling prey to its own store of weapons, as well as fires and vandals over the years. But in addition to the castle, Bannerman built a residence on the island, which still stands today. (It's not in pieces, since it wasn't packed full of explosives. Visitors are allowed to go inside the house nowadays, which is pretty cool.) There also used to be a promenade around the island made from sunken barges. (For a sense of what it looked like, check out this picture of the island from the National Archives.)
Bannerman also invented a family crest for himself, since he thought he should have one, if he also had a castle. Sure, why not?
[^1] Chen, David W. “Long Abandoned, an Island in the Hudson is Restored.” New York Times, Late Edition (East Coast) ed., Nov 28 1999, p. 1, 45:3. ProQuest. Web. 13 Sep. 2021 .
This post doesn’t link to sources as comprehensively as usual, because it's based on an old episode of Buried Secrets Podcast. I wrote this based on the original episode notes, which I penned when I was worse at adding specific in-line citations. But all of the sources I used are linked at the bottom of the episode shownotes page.