Exercise 35a.

Show that a nonempty subset $H$ of a group $G$ is a subgroup of $G$ if and only if $a, b \in H$ implies $ab^{-1} \in H$.

---

Proof

To prove this statement, which is an equivalence, we need to prove both directions:

$"\Rightarrow"\colon$ If $H$ is a subgroup of $G$ then if $a, b \in H$ implies $ab^{-1} \in H$
$"\Leftarrow"\colon$ If $a, b \in H$ implies $ab^{-1} \in H$, then $H$ is a subgroup of $G$

Proof of $"\Rightarrow"$:

We assume $H$ is subgroup of $G$ and we need to show $a, b \in H$ implies $ab^{-1} \in H$.

Since $H$ is a subgroup, we have that the following is true for all $x,y \in H$:

$$x \in H \implies x^{-1} \in H \quad (1)$$ $$x, y \in H \implies xy \in H \quad (2)$$

Let $a, b \in H$. By $(1)$ we have $b^{-1} \in H$, that is $a, b^{-1} \in H$. Now, by $(2)$, this implies that $ab^{-1} \in H$, which is what we needed to show.

Proof of $"\Leftarrow"$:

Now we assume $a, b \in H$ implies $ab^{-1} \in H$. We want to show that $H$ is a subgroup of $G$. To do this, we need to show that:

(1) $a \in H \implies a^{-1} \in H$
(2) $a, b \in H \implies ab \in H$

Proof of (1): $a \in H \implies a^{-1} \in H$

Let $a \in H$. By our assumption, for $b = a$, we get:

$$aa^{-1} \in H,$$

and because $aa^{-1} = e$ we have shown that the identity element $e$ is in $H$. Now, since $e, a \in H$, again using the assumption, we get:

$$ea^{-1} \in H$$

Thus, we have shown that $a \in H \implies a^{-1} \in H$, and the proof of (1) is complete.

Proof of (2): $a, b \in H \implies ab \in H$

Let $a, b \in H$. By (1), we know that $b \in H \implies b^{-1} \in H$. Now, since $a, b^{-1} \in H$, by the starting assumption, we get:

$$a(b^{-1})^{-1} \in H$$

and since $a(b^{-1})^{-1} = ab$, we have that $ab \in H$. Thus, we have shown that $a, b \in H \implies ab \in H$, and the proof of (2) is complete.

Since (1) and (2) are true, $H$ is indeed a subgroup of $G$, and the statement is proven.

We proved both directions, the proof is complete.

---

Additional thoughts

I had one doubt with this proof from which I learned something. I started this exercise just playing around with the assumptions. The "$\implies$" direction was very straightforward. For the "$\Longleftarrow$" direction I again just started playing with the given assumption, trying to see if something will pop up. Along the way I noticed I could prove $a^{-1} \in H$ if $e, a \in H$. But to do that I needed to show that $e$ is in $H$. Then as I was moving symbols around, I realised that I can set $b = a$ and use the assumption to show that $e \in H$. While I was pretty sure I can set $b=a$ due to my prior experience with proofs, I still had some doubts. I thought, there is no $b \neq a$ condition, so who says I can't put $b = a$? I found this answer on MSE, here they brought to my attention the actual name of this rule of inference that allows us to to set $a = x, b = y$ for whatever $x, y \in H$.