Consider an engineer that needs to design a bridge.
The engineer will determine the required strength of the bridge by estimating its typical usage. Naturally, if the bridge is too weak, it will collapse and people will be seriously injured.
The engineer may estimate that 10 trucks and 40 cars will generally be on the bridge at a given time. However, what happens in the rare and freakish event that 40 trucks all drive through at the same time?
Well, if the engineer designed it for the typical usage, the bridge will collapse under the rare event.
Thus, a 'margin of safety' is needed to prevent disasters for these rare occasions.
'Margin of safety' is the difference between its designed level of strength and its ultimate level of strength. In other words, it's a safety buffer between the expected value and the maximum value.
The margin of safety is particularly needed when all factors and variables cannot be known. It's used as a safeguard for errors and the higher the uncertainty, the higher the required margin of safety. 
In the engineering context, another name for margin of safety is 'factor of safety'.
A Major Limitation:
Increased cost and wastage.
As the factor of safety increases, dimensions of the component, material requirement and cost increase. The factor of safety is low for cheap machine parts. 
Another example: If a person allowed a 10 minute margin of safety while travelling, they would have wasted that 10 minutes on something that could have been spent elsewhere.
Now, back to the benefits.
Some great examples of Margin of Safety:
1. Designing an Internet Service
For example, designing an internet service that can support 1000 users is straightforward.
However, to account for unanticipated users of the service (e.g., downloading large files), the design specification can be multiplied by a factor (e.g. three).
In this case, a safety factor of three would mean that the service would be rated to support 1000 users, but actually designed to support three times that many, or three thousand users. 
2. In Investing
If you feel that a stock is worth $10, buying it at $7.50 will give you a margin of safety in case your analysis turns out to be incorrect and the stock is really only worth $9. 
3. In Accounting
Margin of safety in accounting context is the difference between the sales you expect to make vs the sales you need to break even.
For example, imagine that you expect to make $1000 in sales this year. Your costs of running the business is only $800 dollars. This leaves $200.
You can breathe now, you’re safe! 
5. For Pharmaceutical Drugs
An important concept for creating new drugs for therapy, as well as assessing probability of serious side effects for drugs of abuse, is called the margin of safety.
The margin of safety is determined by the doses necessary to cause the intended (therapeutic or recreational) effects and the toxic unattended affects. The larger the margin of safety, the less likely that serious adverse side effects will occur when using the drug to treat medical problems or even when abusing it.
Drugs with relatively narrow margins of safety, such as phencyclidine (PCP) or cocaine, have very high rate of serious reactions in populations who abuse these substances. 
6. Some Trivial Life Examples
I live in Orlando, Florida, and one of the main attractions besides the Disney parks and Universal Studios is a water park called "Wet ‘N’ Wild."
This water park has a giant water slide, and a person must wait until the previous “slider” has gone down the slide before he begins his descent.
This is closely governed by the slide attendants to ensure a good “margin of safety.”
We all know about a margin of safety when we drive.
If a person is tailgating a driver at sixty miles an hour in rush hour traffic, there is little margin for error!
Having sufficient "margin" is necessary in every area of life. 
7. In Construction
Builders can sometimes take shortcuts shortcuts which can create problems. In the example below, the author is referring to the need to factor in a 'margin of safety', even in the construction process itself.
Furthermore, who is to say how careful the workmen might be in attaching the heavy weight to the end of the completed cantilever? How many of them will stand on the beam during the construction process, thus adding considerable weight beyond that of the boulder being hung from the end? Will they place it gently on the hook, or will they drop it with such force that it will endanger the beam, [similar to] the way a sudden blow does a pane of glass? And what about the weight of the beam itself?
To cover the many contingencies that might overload an understrength beam, engineers often divide the predicted weight it can support by as much as a factor of six or eight.
Such conservatisms are known as factors of safety… 
* * *
 Design of Machine Elements by V. B. Bhandari
 Universal Principles of Design, Revised and Updated: 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions, and Teach Through Design by William Lidwell
 Invention by Design: How Engineers Get from Thought to Thing by Henry Petroski
 Roark’s Formulas for Stress and Strain, 7th Ed by Warren C. Young, Richard G Budynas
 Managerial Accounting: Tools for Business Decision Making by Jerry J. Weygandt, Paul D. Kimmel, Donald E. Kieso
 Drugs and Society by Glen R. Hanson, Peter J. Venturelli, Annette E. Fleckenstein
 Stress Less by Don Colbert
Factor of safety is represented as a ratio. However, I have left this out as conceptually this is unimportant.
In the aeronautical engineering context, margin of safety uses a different formula to factor of safety. However, I have left this out as conceptually this is unimportant. 
Founder: Margin of safety was popularised by Benjamin Graham
Categorisation: engineering → quality control
* * *