There was a bit of natter recently (in geological timescale) about Tilly Whim caves and the ban.
My pal Mike Image and I climbed extensively in 1964 on these cliffs. We put put a few new routes immediately below the castle. Access was by climbing over the wall and abseiling initially and then a route to climb down was found (we called it Florence, after the rented boat we hired to recce from).
While doing the recce and trying to get onto the cliff from the boat we heard the sound of a ship's bridge telegraph (that's how close it was!). It was a navy destroyer monitoring the Cowes to Torquay power boat race. Presumably they said something fairly unflattering about those two twerps. Fair enough. Anyway over several visits we put up some decent routes which can be found in the Climbers Club guide book, "Dorset" published in 1969. The ban came into force soon after but not because of us. We were low profile. We slept in the caves regularly, indeed it was a second home that year.
The ban was to discourage "grockles" (lovely old word that dates folk) from scrambling and getting into trouble, especially children.
Under certain circumstances the owners of the castle granted access if one "crawled graciously" enough.