16.863J | Spring 2016 | Graduate

System Safety

Assignments and Class Project

Assignment 2: How Safe is Safe Enough?

[Safeware] = Leveson, Nancy G. Safeware: System Safety and Computers. Addison-Wesley Professional, 1995. ISBN: 9780201119725.

(Note there are no right or wrong answers to these questions. They are meant only to help you to think about these issues and to start to formulate your own opinions.)

  1. Chapter 1 of [Safeware] argues for a shift of responsibility for controlling risk from individuals and personal responsibility to organizational or public responsibility. 

    1. Do you agree with this argument? (You are allowed to disagree with me)
    2. Personal accountability tends to lessen as professional roles become narrowly differentiated. Consider the following reasonable and common views of professional roles today:
    3. Consider the following accident:
    4. Most professional societies have an ethical code. What is the ethical code for the professional society in your industry?
  2. Consider the following two quotes:

The excerpt from “Court Upholds Controversial Regulations.” Science 213, no. 4504 (1981): 185. All rights reserved. This content is excluded from our Creative Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/:

“The Supreme Court in the cotton dust decision on 17 June, says explicitly that OSHA must ignore the results of any cost-benefit comparison when setting a standard for worker exposure to a hazardous substance. Justice William Brennan, writing for the court’s five-person majority, said that “Congress itself decided the basic relationship between costs and benefits by placing the benefit of the worker’s health above all other considerations when it wrote the law in 1970. Yet the agency cannot require exposure controls that are impossible to achieve, nor can it bankrupt an entire industry.” He concluded consideration of anything besides these questions would be inconsistent with Congress’s direction.”

And next is another excerpt from Lockhart, T. W. “Safety Engineering and the Value of Life.” IEEE Technology and Society 9, no. 1 (1981): 3.

“…there is an honored tradition in moral philosophy, associated primarily with Immanuel Kart, according to which human beings have a worth that is not commensurate with that of mere objects. According to this view, because of this incommensurability we must recognize and respect the liberty and dignity of each person and refrain from treating him merely as a means to some end. Human beings may not be used in order to achieve some higher good, for there is no higher good. Let us call this view the Incommensurability Principle.

The Incommensurability Principle has had a powerful appeal for many. This has been true mainly because it has been felt that unless it, or something like it, is accepted, it is not possible to account for such fundamental human rights as the right not to be killed, the right not to have one’s liberty abridged without just cause, and the right to be treated fairly and honestly. The Incommensurability Principle is clearly incompatible with an attempt to place a monetary value on human life or to justify actions on the basis of such a valuation. There is thus further reason for doubting the wisdom of any such attempt. …”

Is it possible to reconcile the Incommensurability Principle with the view that considerations of safety must be weighed against economic costs (the use of cost / risk-benefit analysis)? The Ford Pinto case is an example of the principle involved here. How does this relate to the Cotgrove ideas presented at the end of Chapter 1 in Safeware about differing value systems?

Read the newspaper article about Sunstein for a recent example of this debate. Write (no more than a page or so) about your own preliminary views on these questions above. I realize that these are pretty deep “trans-scientific” questions that you may not have thought about before so what you write may be just your first impressions.

  1. The role of the courts and legal system. Consider the following two cases: 

    • Employers who expose their employees to safety hazards usually escape criminal penalties in the U.S. Victims will often sue companies for damages under tort (i.e., civil law), which allows them to gain compensation without having to prove a crime has been committed. This is true even when people die as a result of horrendous corporate negligence. 

      No example is more shocking than that of the companies in the asbestos industry, especially Johns-Mansville Corporation, which was the largest producer of asbestos. Johns-Manville knew from the 1930s and 1940s onward that asbestos fibers in the lungs cause asbestosis, an incurable form of cancer. For three decades it concealed this information from workers and the public without giving them the right of informed consent to the dangers confronting them. In 1949, Johns-Mansville’s company physician defended a policy of not informing employees diagnosed with asbestosis: “As long as the man feels well, is happy at home and at work and his physical condition remains good, nothing should be said” ( Brodeur, Paul. Outrageous Misconduct: The Asbestos Industry on Trial. Pantheon Books, 1985. ISBN: 9780394533209.) 

      One study showed that 38 percent of asbestos insulation workers die of cancer, 11 percent of them from asbestosis. It has been predicted that “among the twenty-one million living American men and women who had been occupationally exposed to asbestos between 1940 and 1980, there would be between eight and ten thousand deaths from asbestos-related cancer each year for the next twenty years.” The actor Steve McQueen is just one individual included among these statistics. In his youth, he held a summer job handling asbestos insulation and two decades later died of asbestosis. 

      In order to postpone settling the flood of lawsuits, Manville filed for bankruptcy in 1982. (Its assets of $2 billion made it the largest American corporation ever to do so.) A court agreement reached in 1985 allows it to continue operating while paying some $2.5 billion in lawsuits over 25 years.

    • In 1985, for the first time in history, a judge convicted three officials of a company for industrial murder ( Frank, Nancy. Crimes Against Health and Safety. Harrow and Heston, 1985, pp. 21–25. ISBN: 9780911577051.) Film Recovery Systems was a small corporation that recycled silver from used photographic and x-ray plates. Used plates were soaked in a cyanide solution to leach out their silver content. Other companies use this process safely by protecting workers against inhaling cyanide gas and making skin contact with the liquid. Standard safety equipment includes rubber gloves, boots, and aprons, as well as respirators and proper ventilation. None of these precautions were used by Film Recovery Systems. Workers were given useful paper face masks and cloth gloves. Ventilation was terrible, and respirators were not provided. Workers frequently became nauseated and had to go outside to vomit before returning to work at the cyanide vats. This continued until an autopsy on one employee, a Polish immigrant, revealed lethal cyanide poisoning. Charges were brought against the executives of Film Recovery Systems under an Illinois statute, which states that “a person who kills an individual without lawful justification commits murder if, in performing the acts that cause the death … he knows that such acts create a strong probability of death or great bodily harm to the individual or another” (Frank, 1985). During the trial, it was proven that the company president, the plant manager, and the plant foreperson all knew of the dangers of cyanide. They also knew about the hazardous conditions at their plant. Each was sentenced to 25 years in jail and fined $10,000. Critics have disagreed with the conviction on the grounds that murder involves intentional and purposeful killing. At most, say the critics, the executives committed manslaughter, which is killing due to negligence or indifference (such as when drunk drivers kill).

  • Cases in the news very recently include the arrest of the Costa Concordia captain and Mr. Mas who was the president of the French company that put industrial grade silicone in human breast implants.

Questions: How should we assign blame in these contexts? Are there subcategories or distinctions that you think are important? What role should the courts and lawyers play in ensuring public safety?

  1. Individual Responsibility

Approximately 25% of the US population has non-lethal food sensitivity and 1–2% has food allergies severe enough to cause life threatening reactions. Although an individual could be allergic to the proteins in any food, such as fruits, vegetables, and meats, there are eight foods that account for 90% of all food-allergic reactionsi. These are: Milk, egg, peanut, tree nut (walnut, cashew, etc.), fish, shellfish, soy, and wheat. For some consumers, even a trace amount of protein from one of these foods can cause life threatening anaphylactic shock. The de minimus safe level of food protein is not known and experiments to find a safe level are ethically difficult to execute.

Imagine you were an employee in a small food company who is intimately aware of the production practices of this firm. One of your fellow employees comes to you and indicates he heard some other employees discussing a “mistake” regarding allergens made on the night shift. You know the firm has not recovered from the recession and cash is tight. A recall for allergens might be very expensive and put real pressure on the company’s finances, causing the future of the firm to be in doubt. You also know that the regulatory environment is not clear with regard to managing allergens in foods.

What do you think should or should not be done? What are some of the options?

What factors could keep the “right” things from being done? How do you establish the behavior expected by employees if you are a manager?

iFood Anaphylaxis Action Network

Course Info

Learning Resource Types
Lecture Notes
Online Textbook
Written Assignments
Activity Assignments