Personally, I would do it (even if you manage to get it to work, and yes, the built-in configuration mechanism will not allow you). I will always store the behavioral settings of the application with its own application configuration. I did and saw systems where such values were stored in the database, which, of course, is not very possible for the connection string.
The DLL as such is passive (or should be) and acts only on explicit calls from its users and, therefore, should not need any configuration for its own purpose (with the exception of the "global" diagnostic tools, but even then ... ), because the caller can provide it when or before using the appropriate features.
If you have several applications that, by the way, require the same settings, then this is not what the DLL should know or even care.
To make things more manageable, you might consider porting this problem to the build or deployment process. In application configuration files (for example, web.config files), place a placeholder for the connection string. Then during build or deployment (which suits your needs best), replace this placeholder with a true connection string that you can save in a single file.