Keeping things secure is often associated with keeping things secret. "Don't tell anyone how the locking mechanism of the vault works, that will make it harder to break into it". The smart thief preparing a bank heist will of course take one of the engineers who designed the vault out for drinks and get him to spill the beans after a few bottles of something. The idea of keeping things secret to keep things secure is known as 'security by obscurity' and it never works. This is because it is very hard to keep secrets when many people knowing the secret (because they designed the vault for instance, or maintain it, or operate it) are just walking around being their normal human self. People like talking about their work or have a grudge against a former employer or colleague. Obtaining classified information if often a matter of just asking nicely (possibly while pretending to be somebody else). This is known as social engineering.
Because the fact that Keeping Things Secret To Keep Them Secure does not work is so counter-intuitive its almost impossible to eradicate.
When steel vaults became communications devices and computers these old ideas persisted, even though they have been thoroughly disproven time and time again. In 1883 the Dutch cryptographer Auguste Kerckhoffs von Nieuwenhoff published are series of ideas about intrinsically secure information storage and communications by telegraph (the high-tech device of those days). His basic positions that the only secret in in secure system must be they key and all other components must be open for audit to as many experts as possible has been proven over and over again and remains true to this day.
History's lessons
The German Navy apparently did not read von Nieuwenhoffs work because the design of their cryptology device Enigma was based on the premise that its inner workings could be kept a secret from the Allies. This may be possible when there are only two or three devices and all are kept inside military installations but once you start putting hundreds of them on board submarines the chance of one of them being captured goes up steadily.
The story of the capture of enigma by British intelligence and the clever misleading of the Germans by the Allies of the cracking of the Enigma codes is one of they great lesser known stories of how World War-II was won. After misleading the German Intelligence into thinking the submarine U-110 was sunk with the Enigma on board (in reality it was retrieved by the crew of HMS Bulldog) British intelligence was able to keep the Germans convinced that their system was secret and thus secure. The German Navy and Werhmacht kept using the system for several more years while the Allies were reading their mail. This interception and decryption happened pretty much in
real-time thanks to the early computers that were being built by the people at Bletchley Park (aka Station X). The ability to intercept and decrypt most German communications shortened the war by an estimated two years and was key to the success of D-Day. For the Germans of course trusting security-by-obscurity pretty much cost them the battle for the Atlantic and thus the war on the Western front (A great introduction to both the basics of cryptography and the history
of WW-II information warfare is Neal Stephenson's pageturner Cryptonomicon). Since the secret has been out for a while you can download your own paper enigma now.
This rather long winded lead in and history lesson is relevant today because having learnt nothing from all this companies and governments make the same mistakes as the Germans did 65 years ago again and again. And we get stuck with insecure systems that cannot protect us, our information or our money.
Some recent examples of the consequences of this kind of thinking are serious, others are funny.
So can we make systems secure, or at least secure enough? The answer is maybe. It depends on the applications, the acceptable cost and mostly the end-users of the system. More about them here.
Open security, it's the only way
It is now very broadly agreed upon by security experts worldwide that the only way to create reasonably secure systems is to have an open design and development process. This is the exact opposite of the vault manufacturer trying to keep the inner workings of the locking mechanism secret. In an open process all available data on design and the actual implementation of it are shared as quickly as possible with as many experts as possible. This allows all those experts to study both design and implementation and point out possible mistakes and weaknesses to the persons building the system. With many more brains working the problem the end result is generally better than with a few isolated ones working alone.
In software engineering this method has become known as 'Open Source'. This refers to the public availability of the 'sourcecode' of a computerprogram. The 'recepy' to make the actual software. Eric S. Raymond, one of the founders of the Open Source initative formulated in his essay 'The Cathedral and the Bazaar': "given enough eyeballs, all bugs are shallow". The idea being that any software engineering problem van be solved if enough different software developers work on the issue.
What Eric Raymond did was to reformulate a much older method for solving tough problems called the 'scientific method' or 'peer review'. This is the formal method by which scientists keep tabs on each others work and challenge each others thinking. It is by no means a perfect system but over all the scientific method gets results. As a reader you are using dozens of them right now.
Information security, like many scientific problems, is very, very hard. Getting many people to work on the problem with you or for you is still the best way to ensure your system has a fighting chance. As von Nieuwenhoff suggested 125 years ago: the only thing that needs to be secret about an information system is the key one uses to gain acces, the rest should be open to peer review so as to be under permanent scrutiny.
The Open Source Security Testing Methodology Manual (OSSTMM) is a peer-reviewed set of testing methodologies that can be used as a framework for assessing strenghts and weaknesses of information systems, protocols or things like physical buildings.

