If your factory and your classes are in the same assembly, you can tag the internal constructors. This will ensure that no classes outside the assembly can call constructors (without reflection). Your factory, located in the same assembly, sees the constructors as public and, therefore, can access them.
factory . , , , , factory.