Skip to content

Commit c6672ce

Browse files
alexellisalexellis
committed
Expanding on timeouts guide
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
1 parent af6a91a commit c6672ce

2 files changed

Lines changed: 52 additions & 34 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,13 @@
22

33
This is the source repository for the OpenFaaS documentation site.
44

5-
For local development:
5+
For local development
6+
7+
```shell
8+
mkdocs serve
9+
```
10+
11+
Or:
612

713
```shell
814
docker run --rm -it -p 8000:8000 -v `pwd`:/docs squidfunk/mkdocs-material:latest

‎docs/tutorials/expanded-timeouts.md‎

Lines changed: 45 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,11 @@ One of the most common support questions we get is about timeouts. Users often a
88

99
We know it's frustrating, but we would know if there was a regression in OpenFaaS that meant timeouts stopped working as expected.
1010

11+
> For OpenFaaS Edge users should consult the eBook [Serverless For Everyone Else](https://store.openfaas.com/l/serverless-for-everyone-else) and update `docker-compose.yaml`.
12+
1113
## Why do functions timeout?
1214

13-
OpenFaaS functions can run for as long as necessary, some users have reported running executions for 48 hours or longer. The longest executions should be [run asynchronously](), so that the HTTP caller is not blocked waiting for a result.
15+
OpenFaaS functions can run for as long as necessary, some users have reported running executions for 48 hours or longer. The longest executions should be [run asynchronously](/reference/async/), so that the HTTP caller is not blocked waiting for a result.
1416

1517
Functions timeout due to one of the following:
1618

@@ -23,58 +25,68 @@ Once you've followed all the instructions in this guide, make sure you've ruled
2325

2426
You can use the [following GitHub repository](https://github.com/alexellis/go-long) with three sample functions made with Go, Python and Node to confirm the issue isn't in your own function or code.
2527

26-
## Part 1 - the core components
28+
The [openfaas helm chart](https://github.com/openfaas/faas-netes/tree/master/chart/openfaas) contains a detailed explanation of values for each OpenFaaS component including timeouts.
29+
30+
The most recent versions of OpenFaasS Standard and OpenFaaS For Enterprises, only need timeout values to be applied to the Gateway itself, they are then applied to the operator and built-in queue-worker by default.
31+
32+
For dedicated [queue-worker](https://github.com/openfaas/faas-netes/tree/master/chart/queue-worker) installations, you will need to configure their `upstreamTimeout` value separately in values.yaml
2733

28-
When running OpenFaaS on Kubernetes, it is possible to override the timeout on various components of the OpenFaaS gateway, however it is only usually necessary to set the timeout on the gateway itself.
2934

30-
You can find the various options in the [helm chart README](https://github.com/openfaas/faas-netes/tree/master/chart/openfaas).
35+
## Timeout reference
3136

32-
For faasd users, you'll need to edit the equivalent fields in your `docker-compose.yaml` file, see the eBook [Serverless For Everyone Else](https://store.openfaas.com/l/serverless-for-everyone-else).
37+
The function's `exec_timeout` must always be less than or equal to the gateway's `upstream_timeout`. If the function's timeout is larger than the gateway's, the gateway will cancel the request before the function finishes, resulting in a 502 or timeout error.
3338

34-
We need to set the following in the Helm chart's values.yaml file for the OpenFaaS chart:
39+
| Component | Variable | Set via |
40+
|-----------|----------|---------|
41+
| Gateway | `upstream_timeout` | Helm `gateway.upstreamTimeout` |
42+
| Gateway | `read_timeout` | Helm `gateway.readTimeout` |
43+
| Gateway | `write_timeout` | Helm `gateway.writeTimeout` |
44+
| Function | `exec_timeout` | `stack.yaml` environment |
45+
| Function | `read_timeout` | `stack.yaml` environment |
46+
| Function | `write_timeout` | `stack.yaml` environment |
3547

36-
* `gateway.upstreamTimeout`
37-
* `gateway.writeTimeout`
38-
* `gateway.readTimeout`
48+
The function's `exec_timeout` must be equal to or shorter than the gateway's `upstream_timeout`. The gateway's `read_timeout` and `write_timeout` must be slightly longer than `upstream_timeout` to give the gateway time to handle the request cleanly.
3949

40-
All timeouts are to be specified in Golang duration format i.e. `1m` or `60s`, or `1m30s`.
50+
Here is a valid configuration for a function that can run for up to 30 minutes - either synchronously or asynchronously.
4151

42-
If using Helm or Argo CD, then add the following to your values.yaml file:
52+
Gateway (Helm values.yaml):
4353

4454
```yaml
4555
gateway:
46-
writeTimeout: 5m1s
47-
readTimeout: 5m1s
48-
upstreamTimeout: 5m
56+
upstreamTimeout: 30m
57+
readTimeout: 30m1s
58+
writeTimeout: 30m1s
4959
```
5060
51-
Note that `upstreamTimeout` must always be lower than `writeTimeout` and `readTimeout`, to allow the gateway to handle the request before the HTTP server cancels the request.
52-
53-
If using arkade, you can run the following:
54-
55-
```bash
56-
export TIMEOUT=5m
57-
export SERVER_TIMEOUT=5m2s
61+
Function (stack.yaml):
5862
59-
arkade install openfaas \
60-
--set gateway.upstreamTimeout=$TIMEOUT \
61-
--set gateway.writeTimeout=$SERVER_TIMEOUT \
62-
--set gateway.readTimeout=$SERVER_TIMEOUT
63+
```yaml
64+
functions:
65+
my-function:
66+
environment:
67+
exec_timeout: 30m
68+
read_timeout: 30m1s
69+
write_timeout: 30m1s
6370
```
6471
65-
Once installed with these settings, you can invoke functions for up to `5m` synchronously and asynchronously.
66-
67-
## Part 2 - Your function's timeout
72+
Note: setting `exec_timeout: 10m` on a function when the gateway's `upstream_timeout` is only `5m` will **not** give the function 10 minutes. The gateway will timeout and return an error after 5 minutes.
6873

69-
Now that OpenFaaS will allow a longer timeout, configure your function.
74+
## Configure your function's timeout
7075

71-
OpenFaaS functions usually embed a component called the watchdog, which is responsible for implementing timeouts in a consistent way across different languages. Most templates use the newer of-watchdog, but a few may still be using the classic watchdog for compatibility reasons.
76+
OpenFaaS functions usually embed a component called the watchdog, which is responsible for implementing timeouts in a consistent way across different languages.
7277

73-
For the newer templates based upon HTTP which use the of-watchdog, adapt the following sample: [go-long: Golang function that runs for a long time](https://github.com/alexellis/go-long)
78+
Partial example showing the timeouts:
7479

75-
For classic templates using the classic watchdog, you can follow the workshop: [Lab 8 - Advanced feature - Timeouts](https://github.com/openfaas/workshop/blob/master/lab8.md)
80+
```yaml
81+
functions:
82+
job:
83+
environment:
84+
write_timeout: 5m1s
85+
read_timeout: 5m1s
86+
exec_timeout: 5m
87+
```
7688

77-
If you're unsure which template you're using, check the source code of the Dockerfile in the `templates` folder when you build your functions, you should see a `FROM` line at the top of the file that will specify one or the other.
89+
For an example of Node, Python and Golang, see the: [go-long samples](https://github.com/alexellis/go-long).
7890

7991
## Load Balancers, Ingress, and service meshes
8092

0 commit comments

Comments
 (0)