Use httpserver.IndexFile() to determine index files

Test if middleware pushes indexfile when requesting directory

Fix codereview issues

Serve original request first, push later

Revert "Serve original request first, push later"

This reverts commit 2c66f01115747e5665ba7f2d33e2fd551dc31877.

log,error: fix roller parser issue

Signed-off-by: Tw <tw19881113@gmail.com>

Include test for iOS 11 beta

* Revert fastcgi to emove persistent connections.

* Fix linting errors

* reintroduce timeout tests

* check for non-zero timeout

* ensure resp is not nil

* Adding nobots directive

* Moving nobots directive behind log one.

* Move nobots directive to a better position

gzip: change ETag to weak etag after gzip

According to https://tools.ietf.org/html/rfc7232#section-2.1
> Likewise, a validator is weak if it is shared by two or more
representations of a given resource at the same time, unless those
representations have identical representation data.  For example, if
the origin server sends the same validator for a representation with
a gzip content coding applied as it does for a representation with no
content coding, then that validator is weak.

Therefore, after gzip, we should change the original etag to weak etag.

Not much changed but it's a clean pull now; easier for future reference.

* uses more of the builtin functionality for starting and stopping of the process by using command and command_args along with procname
* removed -f from daemon as this was hiding error message that were sent to stdout on startup, now writing stdout to the logfile directly

for example, this was being hidden:

“Activating privacy features.. [www.domain.com] failed to get certificate: Error presenting token: Could not find the start of authority”

it now shows up in the log

* aded “caddy_env” to allow the setting of environment variables that caddy might need, for example when setting creds for “DNS Challenge”

* added a check to ensure caddy_config_path file exists

* rename requestid request_id

* rename folder

* folder name match package name requestid

proxy: add Header load balancing policy

gzip,mime: show response header instead of the request one in test message

adds unix timestamp placeholder

proxy: Fix CI fail from format token in Error call

go vet caused a build fail in https://travis-ci.org/mholt/caddy/jobs/248392875:
   upstream_test.go:480::error: possible formatting directive in Error call (vet)

This patch changes the Error call added in commit 078c9915 to
an Errorf call to support the use of the %d token.

markdown: fix the real ineffectual assignments in test

Signed-off-by: Tw <tw19881113@gmail.com>

* vendor: update Lumberjack dep

* httpserver/roller: introduce rotate_compress directive

This directive will enable gzip compression provided by [Lumberjack](https://github.com/natefinch/lumberjack/pull/43).

The directive `rotate_compress` can be `true` or `false`, being `false` by default.

* httpserver/roller: remove need to set bool with rotate_compress option

markdown: reload template on each request and fix fake tests

Signed-off-by: Tw <tw19881113@gmail.com>

See discussion on commit c9b022b5

If we overwrite the Server header, it becomes difficult/impossible to
know from the client whether the request was proxied through Caddy.

Signed-off-by: Tw <tw19881113@gmail.com>

Signed-off-by: Tw <tw19881113@gmail.com>

* Create request_id directive #1590

* Address Comments

* Fix TestListenerAddrEqual

* requestid: Add some tests

* Address Comments by tobya

* Address Comments