Skip to content

Commit

Permalink
smtp server support
Browse files Browse the repository at this point in the history 
GUI at localhost:9999/smtp/gui (via http server)

see #58024
  • Loading branch information
@martinrode
Lucas Hinderberger authored and martinrode committed Jul 12, 2024
1 parent 2ca2a7d commit 4bc7e61
Show file tree
Hide file tree
Showing 26 changed files with 3,223 additions and 47 deletions.
283 changes: 283 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ This starts the command with the following default settings:
| --- | --- |
| `stop-on-fail` | Stop execution of later test suites if a test suite fails |

### Keep running

- `keep-running`: Wait for a keyboard interrupt after each test suite invocation.
This can be useful for keeping the HTTP / SMTP server for manual inspection.

### Configure logging

Per default request and response of a request will be logged on test failure. If you want to see more information you
Expand Down Expand Up @@ -181,6 +186,12 @@ Manifest is loaded as **template**, so you can use variables, Go **range** and *
"testmode": false
},

// Optional temporary SMTP Server (see below)
"smtp_server": {
"addr": ":9025",
"max_message_size": 1000000,
},

// Specify a unique log behavior only for this single test.
"log_network": true,
"log_verbose": false,
Expand Down Expand Up @@ -2855,3 +2866,275 @@ The expected response:
}
}
```

## SMTP Server
### Summary and Configuration
The apitest tool can run a mock SMTP server intended to catch locally sent
emails for testing purposes.

To add the SMTP Server to your test, put the following in your manifest:

```yaml
{
"smtp_server": {
"addr": ":9025", // address to listen on
"max_message_size": 1000000 // maximum accepted message size in bytes
// (defaults to 30MiB)
}
}
```

The server will then listen on the specified address for incoming emails.
Incoming messages are stored in memory and can be accessed using the HTTP
endpoints described further below. No authentication is performed when
receiving messages.

If the test mode is enabled on the HTTP server and an SMTP server is also
configured, both the HTTP and the SMTP server will be available during
interactive testing.

### HTTP Endpoints
On its own, the SMTP server has only limited use, e.g. as an email sink for
applications that require such an email sink to function. But when combined
with the HTTP server (see above in section [HTTP Server](#http-server)),
the messages received by the SMTP server can be reproduced in JSON format.

When both the SMTP server and the HTTP server are enabled, the following
additional endpoints are made available on the HTTP server:

#### /smtp/gui
A very basic HTML/JavaScript GUI that displays and auto-refreshes the received
messages is made available on the `/smtp/gui` endpoint.

#### /smtp
On the `/smtp` endpoint, an index of all received messages will be made
available as JSON in the following schema:

```json
{
"count": 3,
"messages": [
{
"from": [
"testsender@programmfabrik.de"
],
"idx": 0,
"isMultipart": false,
"receivedAt": "2024-07-02T11:23:31.212023129+02:00",
"smtpFrom": "testsender@programmfabrik.de",
"smtpRcptTo": [
"testreceiver@programmfabrik.de"
],
"to": [
"testreceiver@programmfabrik.de"
]
},
{
"from": [
"testsender2@programmfabrik.de"
],
"idx": 1,
"isMultipart": true,
"receivedAt": "2024-07-02T11:23:31.212523916+02:00",
"smtpFrom": "testsender2@programmfabrik.de",
"smtpRcptTo": [
"testreceiver2@programmfabrik.de"
],
"subject": "Example Message",
"to": [
"testreceiver2@programmfabrik.de"
]
},
{
"from": [
"testsender3@programmfabrik.de"
],
"idx": 2,
"isMultipart": false,
"receivedAt": "2024-07-02T11:23:31.212773829+02:00",
"smtpFrom": "testsender3@programmfabrik.de",
"smtpRcptTo": [
"testreceiver3@programmfabrik.de"
],
"to": [
"testreceiver3@programmfabrik.de"
]
}
]
}
```

Headers that were encoded according to RFC2047 are decoded first.

#### /smtp/$idx
On the `/smtp/$idx` endpoint (e.g. `/smtp/1`), metadata about the message with
the corresponding index is made available as JSON:

```json
{
"bodySize": 306,
"contentType": "multipart/mixed",
"contentTypeParams": {
"boundary": "d36c3118be4745f9a1cb4556d11fe92d"
},
"from": [
"testsender2@programmfabrik.de"
],
"headers": {
"Content-Type": [
"multipart/mixed; boundary=\"d36c3118be4745f9a1cb4556d11fe92d\""
],
"Date": [
"Tue, 25 Jun 2024 11:15:57 +0200"
],
"From": [
"testsender2@programmfabrik.de"
],
"Mime-Version": [
"1.0"
],
"Subject": [
"Example Message"
],
"To": [
"testreceiver2@programmfabrik.de"
]
},
"idx": 1,
"isMultipart": true,
"multiparts": [
{
"bodySize": 15,
"contentType": "text/plain",
"contentTypeParams": {
"charset": "utf-8"
},
"headers": {
"Content-Type": [
"text/plain; charset=utf-8"
]
},
"idx": 0,
"isMultipart": false
},
{
"bodySize": 39,
"contentType": "text/html",
"contentTypeParams": {
"charset": "utf-8"
},
"headers": {
"Content-Type": [
"text/html; charset=utf-8"
]
},
"idx": 1,
"isMultipart": false
}
],
"multipartsCount": 2,
"receivedAt": "2024-07-02T12:54:44.443488367+02:00",
"smtpFrom": "testsender2@programmfabrik.de",
"smtpRcptTo": [
"testreceiver2@programmfabrik.de"
],
"subject": "Example Message",
"to": [
"testreceiver2@programmfabrik.de"
]
}
```

Headers that were encoded according to RFC2047 are decoded first.

#### /smtp/$idx/body
On the `/smtp/$idx/body` endpoint (e.g. `/smtp/1/body`), the message body
(excluding message headers, including multipart part headers) is made availabe
for the message with the corresponding index.

If the message was sent with a `Content-Transfer-Encoding` of either `base64`
or `quoted-printable`, the endpoint returns the decoded body.

If the message was sent with a `Content-Type` header, it will be passed through
to the HTTP response.

#### /smtp/$idx/multipart
For multipart messages, the `/smtp/$idx/multipart` endpoint (e.g.
`/smtp/1/multipart`) will contain an index of that messages multiparts in the
following schema:

```json
{
"multiparts": [
{
"bodySize": 15,
"contentType": "text/plain",
"contentTypeParams": {
"charset": "utf-8"
},
"headers": {
"Content-Type": [
"text/plain; charset=utf-8"
]
},
"idx": 0,
"isMultipart": false
},
{
"bodySize": 39,
"contentType": "text/html",
"contentTypeParams": {
"charset": "utf-8"
},
"headers": {
"Content-Type": [
"text/html; charset=utf-8"
]
},
"idx": 1,
"isMultipart": false
}
],
"multipartsCount": 2
}
```

#### /smtp/$idx[/multipart/$partIdx]+
On the `/smtp/$idx/multipart/$partIdx` endpoint (e.g. `/smtp/1/multipart/0`),
metadata about the multipart with the corresponding index is made available:

```json
{
"bodySize": 15,
"contentType": "text/plain",
"contentTypeParams": {
"charset": "utf-8"
},
"headers": {
"Content-Type": [
"text/plain; charset=utf-8"
]
},
"idx": 0,
"isMultipart": false
}
```

Headers that were encoded according to RFC2047 are decoded first.

The endpoint can be called recursively for nested multipart messages, e.g.
`/smtp/1/multipart/0/multipart/1`.

#### /smtp/$idx[/multipart/$partIdx]+/body
On the `/smtp/$idx/multipart/$partIdx/body` endpoint (e.g.
`/smtp/1/multipart/0/body`), the body of the multipart (excluding headers)
is made available.

If the multipart was sent with a `Content-Transfer-Encoding` of either `base64`
or `quoted-printable`, the endpoint returns the decoded body.

If the message was sent with a `Content-Type` header, it will be passed through
to the HTTP response.

The endpoint can be called recursively for nested multipart messages, e.g.
`/smtp/1/multipart/0/multipart/1/body`.
31 changes: 28 additions & 3 deletions api_testsuite.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"net/http"
"net/url"
"os"
"os/signal"
"path/filepath"
"strconv"
"sync"
Expand All @@ -17,6 +18,7 @@ import (
"github.com/sirupsen/logrus"

"github.com/programmfabrik/apitest/internal/httpproxy"
"github.com/programmfabrik/apitest/internal/smtp"
"github.com/programmfabrik/apitest/pkg/lib/datastore"
"github.com/programmfabrik/apitest/pkg/lib/filesystem"
"github.com/programmfabrik/apitest/pkg/lib/report"
Expand All @@ -34,6 +36,10 @@ type Suite struct {
Testmode bool `json:"testmode"`
Proxy httpproxy.ProxyConfig `json:"proxy"`
} `json:"http_server,omitempty"`
SmtpServer *struct {
Addr string `json:"addr"`
MaxMessageSize int64 `json:"max_message_size"`
} `json:"smtp_server,omitempty"`
Tests []any `json:"tests"`
Store map[string]any `json:"store"`

Expand All @@ -48,12 +54,13 @@ type Suite struct {
reporterRoot *report.ReportElement
index int
serverURL *url.URL
httpServer http.Server
httpServer *http.Server
httpServerProxy *httpproxy.Proxy
httpServerDir string
idleConnsClosed chan struct{}
HTTPServerHost string
loader template.Loader
smtpServer *smtp.Server
}

// NewTestSuite creates a new suite on which we execute our tests on. Normally this only gets call from within the apitest main command
Expand Down Expand Up @@ -175,11 +182,15 @@ func (ats *Suite) Run() bool {
logrus.Infof("[%2d] '%s'", ats.index, ats.Name)
}

ats.StartSmtpServer()
defer ats.StopSmtpServer()

ats.StartHttpServer()
defer ats.StopHttpServer()

err := os.Chdir(ats.manifestDir)
if err != nil {
logrus.Errorf("Unable to switch working directory to %q", ats.manifestDir)
logrus.Fatalf("Unable to switch working directory to %q", ats.manifestDir)
}

start := time.Now()
Expand Down Expand Up @@ -220,7 +231,21 @@ func (ats *Suite) Run() bool {
}
}

ats.StopHttpServer()
if keepRunning { // flag defined in main.go
logrus.Info("Waiting until a keyboard interrupt (usually CTRL+C) is received...")

if ats.HttpServer != nil {
logrus.Info("HTTP Server URL:", ats.HttpServer.Addr)
}
if ats.SmtpServer != nil {
logrus.Info("SMTP Server URL:", ats.SmtpServer.Addr)
}

sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt)

<-sigChan
}

return success
}
Expand Down
2 changes: 2 additions & 0 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
github.com/Masterminds/sprig/v3 v3.2.3
github.com/PuerkitoBio/goquery v1.8.1
github.com/clbanning/mxj v1.8.4
github.com/emersion/go-smtp v0.21.2
github.com/mattn/go-sqlite3 v1.14.17
github.com/moul/http2curl v1.0.0
github.com/pkg/errors v0.9.1
Expand All @@ -32,6 +33,7 @@ require (
github.com/antchfx/xmlquery v1.3.18 // indirect
github.com/antchfx/xpath v1.2.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
github.com/gofrs/uuid v4.4.0+incompatible // indirect
Expand Down
Loading

0 comments on commit 4bc7e61

Please sign in to comment.