The Law of Total Probability, introduction to probability lesson

A factory takes 60 percent of its items from source A and 40 percent from source B, and an item is faulty with probability $0.02$ if it comes from A and $0.05$ if it comes from B. What is the overall probability that an item is faulty? The plain average of the two rates is $0.035$ , but source A supplies more of the items, so its rate should count for more. The law of total probability says exactly how to combine rates measured inside separate groups into one overall probability. This lesson states the law, explains why each rate is weighted by the probability of its group, and shows how to handle a group of probability zero.

Key terms

Partition

Definition Events $B_1, \dots, B_k$ form a partition of the sample space $\Omega$ if each $B_i$ is nonempty and the events are pairwise disjoint, $B_i \cap B_j = \varnothing$ for $i \ne j$ , and exhaustive, $B_1 \cup \cdots \cup B_k = \Omega$ . Every outcome then lies in exactly one of the groups. For any event $E$ , the pieces $E \cap B_1, \dots, E \cap B_k$ are disjoint and their union is $E$ .

In plain words A partition gives every item exactly one label. No item is left out, and no item gets two labels.

Example In the factory model let $A$ be the event that an item comes from source A and $B$ the event that it comes from source B. These two events form a partition, because every item has exactly one source. Labels such as tagged $(T)$ and priority $(P)$ fail to form a partition if some item is both or neither. The four combinations $T \cap P$ , $T \cap P^{c}$ , $T^{c} \cap P$ and $T^{c} \cap P^{c}$ are disjoint and cover every item, so the nonempty ones among them form a partition.

Group weight

Definition The weight of a group $B_i$ in a partition is its probability $P(B_i)$ . The weights are nonnegative, and because the groups are disjoint and their union is $\Omega$ , additivity gives

\sum_{i=1}^{k} P(B_i) = P(\Omega) = 1.

In plain words A group weight says how often a randomly selected outcome falls in that group. The weights split the total probability $1$ among the groups.

Example In the factory model the weights are $P(A) = 0.6$ and $P(B) = 0.4$ , and $0.6 + 0.4 = 1$ . A different supply contract might change them to $0.2$ and $0.8$ , which still add to $1$ .

Conditional rate

Definition For a target event $E$ and a group $B_i$ with $P(B_i) > 0$ , the conditional rate of $E$ in that group is

P(E \mid B_i) = \frac{P(E \cap B_i)}{P(B_i)}.

It is a probability measured with $B_i$ as the reference group. Rates of different groups refer to different reference groups, so they need not add to $1$ . The rate $P(E \mid B_i)$ differs from the joint probability $P(E \cap B_i)$ unless $P(B_i) = 1$ or $P(E \cap B_i) = 0$ .

In plain words A conditional rate says how often the event happens among the items of one group. The weight says how large the group is, and the rate says what happens inside it.

Example Let $F$ be the event that an item is faulty. The model gives $P(F \mid A) = 0.02$ and $P(F \mid B) = 0.05$ . These rates add to $0.07$ , which is no contradiction, since they describe different groups. The joint probability that an item is faulty and from source A is $P(F \cap A) = 0.6 \cdot 0.02 = 0.012$ , smaller than the rate $0.02$ because only 60 percent of the items come from A.

Law of total probability

Definition Let $B_1, \dots, B_k$ be a partition of $\Omega$ and let $E$ be any event. Because the pieces $E \cap B_i$ are disjoint with union $E$ ,

P(E) = \sum_{i=1}^{k} P(E \cap B_i).

By the multiplication rule, each group with $P(B_i) > 0$ contributes $P(E \cap B_i) = P(B_i)\,P(E \mid B_i)$ . So when every group has positive probability,

P(E) = \sum_{i=1}^{k} P(B_i)P(E \mid B_i).

The same law holds for a partition into countably many events.

In plain words Split the event by group, find each group’s share of it as weight times rate, and add the shares. In a two-stage probability tree that first picks the group and then decides the event, this is the sum over the paths that end in the event.

Example For the factory,

\begin{aligned} P(F) &= 0.6(0.02) + 0.4(0.05) \\[0.4em] &= 0.012 + 0.020 = 0.032. \end{aligned}

A constructed population of 10,000 items that matches the model exactly has 6,000 items from A, of which 120 are faulty, and 4,000 from B, of which 200 are faulty. Then 320 of the 10,000 items are faulty, a share of $0.032$ . These counts only illustrate the model. They are not measured data, and a random batch of that size need not have exactly these counts.

Weighted average

Definition If $w_1, \dots, w_k \ge 0$ and $w_1 + \cdots + w_k = 1$ , the number $w_1 r_1 + \cdots + w_k r_k$ is a weighted average of $r_1, \dots, r_k$ . It lies between the smallest and the largest $r_i$ that carry positive weight. The law of total probability writes $P(E)$ as a weighted average of the conditional rates of the groups with positive weight, with the group weights as the weights. The plain average of the rates is the special case of equal weights.

In plain words A large group pulls the overall probability toward its own rate. The overall answer lands between the lowest and the highest rate among groups with positive weight, which gives a quick check on any calculation.

Example The plain average of $0.02$ and $0.05$ is $0.035$ , but the weights $0.6$ and $0.4$ give $0.032$ . If the mix changes to $0.2$ from A and $0.8$ from B, with the same rates,

\begin{aligned} P(F) &= 0.2(0.02) + 0.8(0.05) \\[0.4em] &= 0.004 + 0.040 = 0.044. \end{aligned}

Both $0.032$ and $0.044$ lie between $0.02$ and $0.05$ . Moving weight to the source with the higher fault rate raised the overall probability, although neither rate changed.

Zero-weight group

Definition A group $B_j$ of a partition with $P(B_j) = 0$ is a zero-weight group. Its joint contribution is $P(E \cap B_j) = 0$ , since $E \cap B_j \subseteq B_j$ , so it can be left out of the sum. With $I$ the set of indices $i$ for which $P(B_i) > 0$ , the law reads

P(E) = \sum_{i \in I} P(B_i)P(E \mid B_i).

The rate $P(E \mid B_j)$ is not defined by the ratio formula, since it would divide by zero. So the sum should not contain a term like $0 \cdot P(E \mid B_j)$ , which multiplies zero by an undefined quantity.

In plain words A group with probability zero contributes nothing to the total, and the ratio formula gives it no rate. Drop it, or keep its joint contribution of $0$ , but do not invent a rate for it.

Example Suppose the supplier list also names a third source C with weight $P(C) = 0$ , while $P(A) = 0.6$ and $P(B) = 0.4$ as before. The events $A$ , $B$ and $C$ still form a partition. The three joint pieces are $P(F \cap A) = 0.012$ , $P(F \cap B) = 0.020$ and $P(F \cap C) = 0$ , so $P(F) = 0.032$ as before. No fault rate for source C enters the calculation, and the model does not define one.

Check yourself

A request goes to the main server with probability $0.7$ and to the backup server with probability $0.3$ . A timeout happens with probability $0.01$ for the main server and $0.08$ for the backup. Find the probability of a timeout and the probability of no timeout.

Show the answer

Let $M$ and $B$ be the events that the request goes to the main and to the backup server, and $T$ the event of a timeout. Each request goes to exactly one server, so $M$ and $B$ form a partition and

\begin{aligned} P(T) &= 0.7(0.01) + 0.3(0.08) \\[0.4em] &= 0.007 + 0.024 = 0.031. \end{aligned}

The probability of no timeout is the complement, $P(T^{c}) = 1 - 0.031 = 0.969$ . As a check, $0.031$ lies between the two rates $0.01$ and $0.08$ .

A colleague reports a timeout probability of $0.045$ for the routing in question 1, found by averaging $0.01$ and $0.08$ . For which routing would that number be correct, and why is it wrong here?

Show the answer

The plain average is the weighted average with equal weights. With routing $0.5$ and $0.5$ ,

\begin{aligned} P(T) &= 0.5(0.01) + 0.5(0.08) \\[0.4em] &= 0.005 + 0.040 = 0.045. \end{aligned}

With two groups and different rates, this is the only routing that gives $0.045$ . If a share $w$ goes to the main server, then $w(0.01) + (1 - w)(0.08) = 0.08 - 0.07w$ , and this equals $0.045$ only for $w = 0.5$ . In question 1 the main server, which has the lower rate, receives most of the requests, so it pulls the overall probability down to $0.031$ .

In a model warehouse, 40 percent of parcels are fragile and 30 percent are oversized. Fragile parcels are damaged with probability $0.05$ and oversized parcels with probability $0.02$ . A clerk computes $0.4(0.05) + 0.3(0.02) = 0.026$ as the overall probability of damage. Why is this not the law of total probability, and what would the law need?

Show the answer

The groups “fragile” and “oversized” do not form a partition. Their weights add to $0.7$ , not $1$ , so at least 30 percent of parcels belong to neither group, and their damage is left out entirely. If some parcels are both fragile and oversized, those parcels also enter both terms.

The law needs disjoint, exhaustive groups, the weight of each group and the damage rate of each group with positive weight. One option is the nonempty combinations of fragile or not and oversized or not. Another is a single classification, such as fragile versus not fragile. The given numbers do not determine the answer.

The Law of Total Probability

Key terms

Partition

Group weight

Conditional rate

Law of total probability

Weighted average

Zero-weight group

Check yourself

How this lesson connects

Builds on

Used later in