Original feature request in forum:
https://forum.caddyserver.com/t/caddy-with-specific-hosts-but-on-demand-tls/1704?u=matt

Before, Caddy obtained certificates for every name it could at startup.
And it would only obtain certificates during the handshake for sites
defined with a hostname that didn't qualify at startup (like
"*.example.com" or ":443"). This made sense for most situations, and
helped ensure that certificates were obtained as early and reliably as
possible.

With this change, Caddy will NOT obtain certificates for hostnames it
knows at startup (even if they qualify) if OnDemand is enabled.

But I think this change generalizes well, because a user who specifies
max_certs is deliberately turning on On-Demand TLS, fully aware of
the consequences. It seems dubious to ignore that config when the user
deliberately put it there. We'll see how this goes.