Spaces:
Sleeping
Sleeping
File size: 5,224 Bytes
61d39e2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 |
# API Tester
A test framework for testing the puter HTTP API and puterjs API.
## Table of Contents
- [How to use](#how-to-use)
- [Workflow](#workflow)
- [Shorthands](#shorthands)
- [Basic Concepts](#basic-concepts)
- [Behaviors](#behaviors)
- [Working directory (`t.cwd`)](#working-directory-t-cwd)
- [Implementation](#implementation)
- [TODO](#todo)
## How to use
### Workflow
All commands below should be run from the root directory of puter.
1. (Optional) Start a backend server:
```bash
npm start
```
2. Copy `example_config.yml` and add the correct values:
```bash
cp ./tools/api-tester/example_config.yml ./tools/api-tester/config.yml
```
Fields:
- url: The endpoint of the backend server. (default: http://api.puter.localhost:4100/)
- username: The username of the user to test. (e.g. `admin`)
- token: The token of the user. (can be obtained by logging in on the webpage and typing `puter.authToken` in Developer Tools's console)
- mountpoints: The mountpoints to test. (default config includes 2 mountpoints: `/` for "puter fs provider" and `/admin/tmp` for "memory fs provider")
3. Run tests against the HTTP API (unit tests and benchmarks):
```bash
node ./tools/api-tester/apitest.js
```
4. (experimental) Run tests against the puter-js client:
```bash
node ./tools/api-tester/apitest.js --puterjs
```
### Shorthands
- Run tests against the HTTP API (unit tests and benchmarks):
```bash
node ./tools/api-tester/apitest.js
```
- Run unit tests against the HTTP API:
```bash
node ./tools/api-tester/apitest.js --unit
```
- Run benchmarks against the HTTP API:
```bash
node ./tools/api-tester/apitest.js --bench
```
- Filter tests by suite name:
```bash
node ./tools/api-tester/apitest.js --unit --suite=mkdir
```
- Filter benchmarks by name:
```bash
node ./tools/api-tester/apitest.js --bench --suite=stat_intensive_1
```
- Stop on first failure:
```bash
node ./tools/api-tester/apitest.js --unit --stop-on-failure
```
- (unimplemented) Filter tests by test name:
```bash
# (wildcard matching) Run tests containing "memoryfs" in the name
node ./tools/api-tester/apitest.js --unit --test='*memoryfs*'
# (exact matching) Run the test "mkdir in memoryfs"
node ./tools/api-tester/apitest.js --unit --test='mkdir in memoryfs'
```
- (unimplemented) Rerun failed tests in the last run:
```bash
node ./tools/api-tester/apitest.js --rerun-failed
```
## Basic Concepts
A *test case* is a function that tests a specific behavior of the backend API. Test cases can be nested:
```js
await t.case('normal mkdir', async () => {
const result = await t.mkdir_v2('foo');
expect(result.name).equal('foo');
await t.case('can stat the created directory', async () => {
const stat = await t.stat('foo');
expect(stat.name).equal('foo');
});
});
```
A *test suite* is a collection of test cases. A `.js` file should contain exactly one test suite.
```js
module.exports = {
name: 'mkdir',
do: async t => {
await t.case('normal mkdir', async () => {
...
});
await t.case('recursive mkdir', async () => {
...
});
}
};
```
## Behaviors
### Working directory (`t.cwd`)
- The working directory is stored in `t.cwd`.
- All filesystem operations are performed relative to the working directory, if the given path is not absolute. (e.g., `t.mkdir('foo')`, `t.cd('foo')`, `t.stat('foo')`, etc.)
- Tests will be run under all mountpoints. The default working directory for a mountpoint is `${mountpoint.path}/{username}/api_test`. (This is subject to change in the future, the reason we include `admin` in the path is to ensure the test user `admin` has write access, see [Permission Documentation](https://github.com/HeyPuter/puter/blob/3290440f4bf7a263f37bc5233565f8fec146f17b/src/backend/doc/A-and-A/permission.md#permission-options) for details.)
- The working directory is reset at the beginning of each test suite, since a test suite usually doesn't want to be affected by other test suites.
- The working directory will be inherited from the cases in the same test suite, since a leaf case might want to share the context with its parent/sibling cases.
```js
module.exports = {
name: 'readdir',
do: async t => {
// t.cwd is reset to /admin/api_test
await t.case('normal mkdir', async () => {
// inherits cwd from parent/sibling cases
await t.case('mkdir in subdir', async () => {
// inherits cwd from parent/sibling cases
});
});
}
};
```
## Implementation
- Test suites are registered in `tools/api-tester/tests/__entry__.js`.
## TODO
- [ ] Reset `t.cwd` if a test case fails. Currently, `t.cwd` is not reset if a test case fails.
- [ ] Integrate apitest into CI, optionally running it only in specific scenarios (e.g., when backend code changes).
|