Given a prime $p$, a group is called residually $p$ if the intersection of its $p$-power index normal subgroups is trivial. A group is called virtually residually $p$ if it has a finite index subgroup which is residually $p$. It is well-known that finitely generated linear groups over fields of characteristic zero are virtually residually $p$ for all but finitely many $p$. In particular, fundamental groups of hyperbolic $3$-manifolds are virtually residually $p$. It is also well-known that fundamental groups of $3$-manifolds are residually finite. In this paper the authors prove a common generalisation of these results: every $3$-manifold group is virtually residually $p$ for all but finitely many $p$. This gives evidence for the conjecture (Thurston) that fundamental groups of $3$-manifolds are linear groups.