Skip to content

Commit

Permalink
improved Readme: updated command line parameters; see #72885
Browse files Browse the repository at this point in the history
  • Loading branch information
Philipp Hempel committed Jul 3, 2024
1 parent 0bc0b2b commit 3bf6f63
Showing 1 changed file with 51 additions and 35 deletions.
86 changes: 51 additions & 35 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,40 +54,50 @@ This starts the command with the following default settings:

### Configure which tests should be run

- `--directory testDirectory` or `-d testDirectory`: Defines which directory should be used for running the tests in it. The tool walks recursively trough all subdirectories and runs alls tests that have a "manifest.json" file in alphabetical order of the folder names. (Depth-First-Search)
- `--single path/to/a/single/manifest.json` or `-s path/to/a/single/manifest.json`: Run only a single test. The path needs to point directly to the manifest file. (Not the directory containing it)
| Parameter | | Description |
| --- | --- | --- |
| `--directory testDirectory` | `-d testDirectory` | Defines which directory should be used for running the tests in it. The tool walks recursively trough all subdirectories and runs alls tests that have a `manifest.json` file in alphabetical order of the folder names. (Depth-First-Search) |
| `--single manifest.json` | `-s manifest.json` | Run only a single test. The path needs to point directly to the manifest file. (Not the directory containing it) |

### Stop on fail

- `stop-on-fail`: Stop execution of later test suites if a test suite fails
| Parameter | Description |
| --- | --- |
| `stop-on-fail` | Stop execution of later test suites if a test suite fails |

### Configure logging

Per default request and response of a request will be logged on test failure. If you want to see more information you
can configure the tool with additional log flags

- `--log-network` / `-n`: Log all network traffic
- `--log-datastore`: Logs datastore operations into datastore
- `--log-verbose` / `v`: `--log-network`, `--log-datastore` and a few additional trace informations
- `--log-short`: Show minimal logs, useful for CI chains
- `--log-timestamp` / `-t`: Log the timestamp of the log message into the console
- `--curl-bash`: Log the request as curl command
- `--limit-request`: Limit the lines of request log output. Configure limit in `apitest.yml`
- `--limit-response`: Limit the lines of response log output. Configure limit in `apitest.yml`
| Parameter | | Description |
| --- | --- | --- |
| `--log-network` | `-n` | Log all network traffic |
| `--log-datastore` | | Logs datastore operations into datastore |
| `--log-verbose` | `-v` | `--log-network`, `--log-datastore` and a few additional trace informations |
| `--log-short` | | Show minimal logs, useful for CI chains |
| `--log-timestamp` | `-t` | Log the timestamp of the log message into the console |
| `--curl-bash` | | Log the request as curl command |
| `--limit-request` | | Limit the lines of request log output. Configure limit in `apitest.yml` |
| `--limit-response` | | Limit the lines of response log output. Configure limit in `apitest.yml` |

You can also set the log verbosity per single testcase. The greater verbosity wins.

### Overwrite config parameters

- `--config subfolder/newConfigFile` or `-c subfolder/newConfigFile`: Overwrites the path of the config file (default "./apitest.yml") with "subfolder/newConfigFile"
- `--server URL`: Overwrites base url to the api
- `--report-file newReportFile`: Overwrites the report file name from the apitest.yml config with "newReportFile"
- `--report-format junit`: Overwrites the report format from the apitest.yml config with "junit"
- `--replace-host [host][:port]`: Overwrites built-in server host in template function "replace_host"
| Parameter | | Description |
| --- | --- | --- |
| `--config newConfigFile` | `-c newConfigFile` | Overwrites the path of the config file (default `./apitest.yml`) with `newConfigFile` |
| `--server URL` | | Overwrites base url to the api |
| `--report-file newReportFile` | | Overwrites the report file name from the `apitest.yml` config with `newReportFile` |
| `--report-format junit` | | Overwrites the report format from the `apitest.yml` config with `junit` |
| `--replace-host [host][:port]` | | Overwrites built-in server host in template function `replace_host` |

### Additional parameters

- `--report-format-stats-group 3`: Sets the number of groups for manifests distrubution when using report format `stats`
| Parameter | Description |
| --- | --- |
| `--report-format-stats-group 3` | Sets the number of groups for manifests distrubution when using report format `stats` |

### Examples

Expand All @@ -97,7 +107,7 @@ You can also set the log verbosity per single testcase. The greater verbosity wi
./apitest --directory apitests --verbosity 2 --report-format junit
```

- Only run a single test **apitests/test1/manifest.json** with **no console output** and save the maschine report to the standard file defined in the apitest.yml
- Only run a single test **apitests/test1/manifest.json** with **no console output** and save the maschine report to the standard file defined in the `apitest.yml`

```bash
./apitest --single apitests/test1/manifest.json --log-console-enable false
Expand Down Expand Up @@ -741,7 +751,7 @@ To set data in custom store, you can use 4 methods:

| | |
| --- | --- |
| `store` on the **manifest.json** top level | the data is set before the session authentication (if any) |
| `store` on the `manifest.json` top level | the data is set before the session authentication (if any) |
| `store_response_qjson` in `authentication.store_response_qjson` | |
| `store` on the **test** level | the data is set before **request** and **response** are evaluated |
| `store_response_qjson` on the test level | the data is set after each **response** (If you want the datestore to delete the current entry if no new one could be found with qjson. Just prepend the qjson key with a `!`. E.g. `"eventId":"!body.0._id"` will delete the `eventId` entry from the datastore if `body.0._id` could not be found in the response json) |
Expand Down Expand Up @@ -1478,7 +1488,7 @@ A single test could look as simple as following:
}
```

> Important: The paths to the external files start with a '@' and are relative to the location of the manifest.json or can be web urls e.g. https://programmfabrik.de/testfile.json
> Important: The paths to the external files start with a `@` and are relative to the location of the `manifest.json` or can be web urls e.g. https://programmfabrik.de/testfile.json
The content of the request and response file are execatly the same as if you would place the json code inline:

Expand Down Expand Up @@ -1538,12 +1548,12 @@ Template Functions are invoked using the tags `{{ }}` and upon returning substit

For a reference see [https://golang.org/pkg/text/template](https://golang.org/pkg/text/template)

| | |
| --- | --- |
| `manifest.json` -> external file | load external file |
| external file -> another file | render template with file parameter `"hello"` |
| another file -> external file | return rendered template `"hello world"` |
| external file -> `manifest.json` | return rendered template |
| | |
| --- | --- |
| `manifest.json` -> external file | load external file |
| external file -> another file | render template with file parameter `"hello"` |
| another file -> external file | return rendered template `"hello world"` |
| external file -> `manifest.json` | return rendered template |

### Example

Expand Down Expand Up @@ -1927,10 +1937,13 @@ The key `-` has a special meaning, it returns the entire custom datastore (not t

## `qjson [path] [json]`

Helper function to extract fields from the 'json'.
- `@path`: string; a description of the location of the field to extract. For array access use integers; for object access use keys. Example: 'body.1.field'; see below for more details
- `@json_string`: string; a valid json blob to be queried; can supplied via pipes from 'datastore idx'
- `@result`: the content of the json blob at the specified path
Helper function to extract fields from the `json`

| Parameter | Type | Description |
| --- | --- | --- |
| `@path` | `string` | a description of the location of the field to extract. For array access use integers; for object access use keys. Example: `body.1.field`; see below for more details |
| `@json_string` | `string` | a valid json blob to be queried; can be supplied via pipes from `datastore idx` |
| `@result` | | the content of the json blob at the specified path |

### Example

Expand All @@ -1940,7 +1953,7 @@ The call
{{ qjson "foo.1.bar" "{\"foo": [{\"bar\": \"baz\"}, 42]}" }}
```

would return `baz`.
would return `baz`.

As an example with pipes, the call

Expand All @@ -1954,10 +1967,13 @@ See [gjson](https://github.com/tidwall/gjson/blob/master/README.md)

## `file_csv [path] [delimiter]`

Helper function to load a csv file.
- `@path`: string; a path to the csv file that should be loaded. The path is either relative to the manifest or a weburl
- `@delimiter`: rune; The delimiter that is used in the given csv e.g. ',' Defaults to ','
- `@result`: the content of the csv as json array so we can work on this data with qjson
Helper function to load a csv file

| Parameter | Type | Description |
| --- | --- | --- |
| `@path` | `string` | A path to the csv file that should be loaded. The path is either relative to the manifest or a weburl |
| `@delimiter` | `rune` | The delimiter that is used in the given csv e.g. `,` Defaults to `,` |
| `@result` | | The content of the csv as json array so we can work on this data with qjson |

The CSV **must** have a certain structur. If the structure of the given CSV differs, the apitest tool will fail with a error

Expand Down

0 comments on commit 3bf6f63

Please sign in to comment.