Add health check timeout parameter

[jbiel]

What does this PR do?

A health check timeout parameter has been added. This allows health checks to set custom timeout periods for backends. This value is currently non-configurable and set to 5 seconds.

If the health timeout is configured to be greater than the interval, an error message is logged and the interval is set to the timeout + 1 second. I did this because I noticed prior art of using sane defaults in the case of invalid configuration (I like this; I appreciate that the daemon doesn't just die.) If there's a better way to handle this or if this isn't desirable I'm open to any necessary changes.

Motivation

An application that we inherited periodically takes > 5 seconds to respond to health check requests and we don't want to consider backends that take longer than 5 seconds to respond unhealthy.

Fixes #3784

More

• Added/updated tests
• Added/updated documentation

Additional Notes

• make test-unit was successful
• make docs-verify was successful and docs look good visually
• I'm unsure of why autogen/gentemplates/gen.go changed so much. I ran go generate with go 1.10.3 and go-bindata 3.4.0.

[ldez]

@jbiel we use /~https://github.com/containous/go-bindata

[jbiel]

Thanks @ldez, I've updated the PR to use that version and it's much cleaner. The Arch package sources go-bindata from /~https://github.com/shuLhan/go-bindata.

[jbdoumenjou]

Hi @jbiel, thank you for your work.
if I am not mistaken, the following tests are to be updated:

• provider/docker/config_container_docker_test.go
• provider/docker/config_container_swarm_test.go
• provider/ecs/config_test.go
• provider/marathon/config_test.go
• provider/mesos/config_test.go
• provider/rancher/config_test.go

[juliens]

Thank you for this PR.

[juliens]

WDYT about just having a warn log and not changing the interval value. I think the +1s is a little bit arbitrary.

[jbiel]

We could do this but I'm unsure of the consequences of potentially having multiple health checks running at once. For example:

• Interval is set to 5s
• Timeout is set to 20s
• Backend is taking 15s to respond

In the above case we'll have up to 3 health checks running against a backend at the same time. Is this acceptable?

[juliens]

In fact, we can't launch new healthcheck if we are still waiting for the previous healthcheck response because the healthcheck is in the same goroutine as the interval.
/~https://github.com/containous/traefik/blob/fdf14cd101821b5ab0d5fb62b5ffaa489ad5eaab/healthcheck/healthcheck.go#L116-L126

[jbiel]

OK - I've removed the code that changes the interval and changed the log to warn level.

[jbiel]

@jbdoumenjou, the PR has been updated to include updates to those tests.

[dtomcej]

Could this also be added to the kubernetes provider as healthcheck-timeout?

[juliens]

LGTM

[dtomcej]

LGTM

[ldez]

LGTM