"All related tables" are not always clear, so this is not an obvious situation. There are tables that have a common column, but may never see each other.
However, this is useful to prevent errors that slip past your primary protection and allow you to enter data that is easily tracked.
They do not help make queries more efficient if you have the appropriate indexes in place, and a good application will filter the input so that it will never be needed. But mistakes do happen, and this is a cheap line of defense.
If you just like to create databases, they cannot worry and refine much if you have a basic relationship between parents and children.
Here is some background -
What happened to foreign keys?
source share