Update README.md
Browse files
README.md
CHANGED
|
@@ -31,7 +31,7 @@ tags:
|
|
| 31 |
|
| 32 |
We present an inverse dynamics model that predicts user input actions from short windows of screen recordings. Given 10 consecutive screenshots, it emits the key presses, mouse clicks, cursor movements and scroll events that are visually implied by the frames.
|
| 33 |
|
| 34 |
-
Please refer to the [
|
| 35 |
|
| 36 |
## Summary
|
| 37 |
|
|
@@ -45,7 +45,7 @@ Please refer to the [IDM blog post](https://pdoom.org/crowd_cast_idm.html) for d
|
|
| 45 |
|
| 46 |
## IDM-eval results
|
| 47 |
|
| 48 |
-
All models are evaluated on [
|
| 49 |
|
| 50 |
| Model | Overall F1 | KeyPress F1 | MouseClick F1 | MouseMove F1 | MouseScroll F1 | MM R² | MM cos_mean | MM cov. |
|
| 51 |
| ---------------- | ---------- | ----------- | ------------- | ------------ | -------------- | --------- | ----------- | ------- |
|
|
|
|
| 31 |
|
| 32 |
We present an inverse dynamics model that predicts user input actions from short windows of screen recordings. Given 10 consecutive screenshots, it emits the key presses, mouse clicks, cursor movements and scroll events that are visually implied by the frames.
|
| 33 |
|
| 34 |
+
Please refer to the [blog post](https://pdoom.org/crowd_cast_idm.html) for details and experiments.
|
| 35 |
|
| 36 |
## Summary
|
| 37 |
|
|
|
|
| 45 |
|
| 46 |
## IDM-eval results
|
| 47 |
|
| 48 |
+
All models are evaluated on [our eval set](https://huggingface.co/datasets/p-doom/idm-eval-set), visibility-filtered to `visible` + `inferable` actions. `MM R²` and `MM cos_mean` include missed MouseMove frames as zero predictions; `MM cov.` is MouseMove recall.
|
| 49 |
|
| 50 |
| Model | Overall F1 | KeyPress F1 | MouseClick F1 | MouseMove F1 | MouseScroll F1 | MM R² | MM cos_mean | MM cov. |
|
| 51 |
| ---------------- | ---------- | ----------- | ------------- | ------------ | -------------- | --------- | ----------- | ------- |
|