Spaces:
Sleeping
Sleeping
rmm
commited on
Commit
·
5eadf93
1
Parent(s):
b330986
docs: added some notes on running the visual tests
Browse files- selenium requires running the app, and then running a second browser
or tab that tests the app.
- docs/dev_notes.md +52 -0
docs/dev_notes.md
CHANGED
|
@@ -44,6 +44,25 @@ mkdocs build -c
|
|
| 44 |
|
| 45 |
# Testing
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
## local testing
|
| 48 |
To run the tests locally, we have the standard dependencies of the project, plus the test runner dependencies.
|
| 49 |
|
|
@@ -74,6 +93,39 @@ pytest --junit-xml=test-results.xml
|
|
| 74 |
pytest --cov-report=lcov --cov=src
|
| 75 |
```
|
| 76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
## CI testing
|
| 79 |
|
|
|
|
| 44 |
|
| 45 |
# Testing
|
| 46 |
|
| 47 |
+
## use of markers
|
| 48 |
+
|
| 49 |
+
The CI runs with `--strict-markers` so any new marker must be registered in
|
| 50 |
+
`pytest.ini`.
|
| 51 |
+
|
| 52 |
+
- the basic CI action runs the fast tests only, skipping all tests marked
|
| 53 |
+
`visual` and `slow`
|
| 54 |
+
- the CI action on PR runs the `slow` tests, but stil excluding `visual`.
|
| 55 |
+
- TODO: a new action for the visual tests is to be developed.
|
| 56 |
+
|
| 57 |
+
Check all tests are marked ok, and that they are filtered correctly by the
|
| 58 |
+
groupings used in CI:
|
| 59 |
+
```bash
|
| 60 |
+
pytest --collect-only -m "not slow and not visual" --strict-markers --ignore=tests/visual_selenium
|
| 61 |
+
pytest --collect-only -m "not visual" --strict-markers --ignore=tests/visual_selenium
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
|
| 66 |
## local testing
|
| 67 |
To run the tests locally, we have the standard dependencies of the project, plus the test runner dependencies.
|
| 68 |
|
|
|
|
| 93 |
pytest --cov-report=lcov --cov=src
|
| 94 |
```
|
| 95 |
|
| 96 |
+
## local testing for visual tests
|
| 97 |
+
|
| 98 |
+
We use seleniumbase to test the visual appearance of the app, including the
|
| 99 |
+
presence of elements that appear through the workflow. This testing takes quite
|
| 100 |
+
a long time to execute and is not yet configured with CI.
|
| 101 |
+
|
| 102 |
+
```bash
|
| 103 |
+
# install packages for app and for visual testing
|
| 104 |
+
pip install ./requirements.txt
|
| 105 |
+
pip install -r tests/visual_selenium/requirements_visual.txt
|
| 106 |
+
```
|
| 107 |
+
|
| 108 |
+
**Running tests**
|
| 109 |
+
The execution of these tests requires that the site/app is running already.
|
| 110 |
+
|
| 111 |
+
In one tab:
|
| 112 |
+
```bash
|
| 113 |
+
streamlit run src/main.py
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
In another tab:
|
| 117 |
+
```bash
|
| 118 |
+
# run just the visual tests
|
| 119 |
+
pytest -m "visual" --strict-markers
|
| 120 |
+
# run in demo mode, using firefox (default is chrome)
|
| 121 |
+
pytest -m "visual" --strict-markers -s browser=firefox --demo
|
| 122 |
+
|
| 123 |
+
# the inverse set:
|
| 124 |
+
pytest -m "not slow and not visual" --strict-markers --ignore=tests/visual_selenium
|
| 125 |
+
|
| 126 |
+
```
|
| 127 |
+
|
| 128 |
+
|
| 129 |
|
| 130 |
## CI testing
|
| 131 |
|