timothythomas90 commited on
Commit
24423a3
·
verified ·
1 Parent(s): 7743aff

Add summary.md, README.md, config.json, training_args.json, model.safetensors

Browse files
Files changed (5) hide show
  1. README.md +37 -0
  2. config.json +16 -0
  3. model.safetensors +3 -0
  4. summary.md +54 -0
  5. training_args.json +9 -0
README.md ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - research-notes
5
+ - prompt-engineering
6
+ ---
7
+
8
+ # Notes on Prompt Engineering
9
+
10
+ ## Repository summary
11
+
12
+ This is an exploratory note for **Prompt Engineering**. It records the intended comparison, likely confounders, and reproducibility requirements before any benchmark result is reported.
13
+
14
+ ## What is covered
15
+
16
+ - the scope of the research question and likely confounders
17
+ - a proposed comparison with matched baselines
18
+ - concrete evaluation context such as task-appropriate public benchmarks named in the main note
19
+ - reproducibility checks, failure modes, and open questions
20
+ - topic-relevant references
21
+
22
+ ## How to read this repository
23
+
24
+ Start with `summary.md` for the full note. Sections labeled as plans or hypotheses should not be interpreted as experimental results. If results are added later, they should include dataset versions, commands, seeds, hardware, and raw logs.
25
+
26
+ ## Scope and limitations
27
+
28
+ The note is intentionally exploratory. It does not claim benchmark improvements, completed ablations, released code, or a trained checkpoint. References and proposed datasets provide a starting point for verification rather than evidence that the study has already been run.
29
+
30
+ ## Files
31
+
32
+ - `summary.md` — primary artifact
33
+ - `README.md` — this documentation
34
+
35
+ ## License
36
+
37
+ Released under **mit**. Review the source-data terms separately when this repository is used with external datasets.
config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "CustomResearchModel"
4
+ ],
5
+ "architecture": "transformer",
6
+ "model_type": "transformer",
7
+ "hidden_size": 192,
8
+ "num_hidden_layers": 8,
9
+ "num_attention_heads": 4,
10
+ "intermediate_size": 768,
11
+ "hidden_act": "gelu",
12
+ "max_position_embeddings": 256,
13
+ "layer_norm_eps": 1e-12,
14
+ "checkpoint_status": "initialization-only",
15
+ "notes": "Untrained checkpoint for smoke tests; no benchmark claim."
16
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1758a04d5fd54449b4eb33bfe859f9d6418d1aa9e3b08166e84259271358c2e4
3
+ size 99808
summary.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Prompt Engineering: Research Notes
2
+
3
+ ## Status
4
+
5
+ Working note / experiment plan. No completed benchmark results are claimed here.
6
+
7
+ ## 1. Scope and motivation
8
+
9
+ This document is a research sketch about controllable multimodal generation and prompt sensitivity. The central question is whether the proposed change improves the target behavior under a matched training and evaluation budget. The note deliberately separates hypotheses from observations so that future results can be added without rewriting the rationale.
10
+
11
+ ## 2. Context
12
+
13
+ Research on prompt engineering often mixes improvements from architecture, data scale, preprocessing, and compute. A useful comparison therefore needs controlled baselines and explicit reporting of resource use. For this topic, the main confound is that automatic metrics may miss hallucination, bias, and prompt brittleness.
14
+
15
+ ## 3. Working hypothesis
16
+
17
+ A focused change to the representation or interaction mechanism may improve task accuracy without increasing deployment cost disproportionately. The hypothesis should be rejected if gains disappear after matching parameter count, data exposure, or tuning budget.
18
+
19
+ ## 4. Proposed approach
20
+
21
+ The first implementation should keep modality-specific preprocessing simple, project inputs into a shared representation space, and isolate the new component behind a small interface. Baselines should include a comparable model without the component and a stronger off-the-shelf reference. Any optimization should be applied to all systems, not only the proposed one.
22
+
23
+ ## 5. Evaluation plan
24
+
25
+ | Dataset | Role | Primary measure |
26
+ |---|---|---|
27
+ | Conceptual Captions | primary evaluation | task accuracy |
28
+ | VQAv2 | transfer / robustness | CLIPScore |
29
+ | Flickr30k | transfer / robustness | human factuality rating |
30
+
31
+ Planned comparisons include a matched-capacity baseline, an ablation that removes the proposed component, and an out-of-domain transfer check. Default training values for the first controlled run are learning rate `5e-05`, batch size `16`, and `5` independent seeds. These are planning values, not claims about a finished experiment.
32
+
33
+ ## 6. Reproducibility checklist
34
+
35
+ - Publish exact split identifiers.
36
+ - Log package versions and hardware.
37
+ - Tune baselines under the same budget.
38
+ - Save both aggregate and per-category metrics.
39
+
40
+ ## 7. Failure modes and responsible use
41
+
42
+ The analysis should report subgroup and category-level failures instead of relying only on a single aggregate score. Particular attention is needed because automatic metrics may miss hallucination, bias, and prompt brittleness. No production use is recommended without task-specific validation, data review, and an assessment of privacy and bias.
43
+
44
+ ## 8. Open questions
45
+
46
+ - How sensitive is the conclusion to preprocessing and random seed?
47
+ - Does the proposed component improve calibration as well as the primary metric?
48
+ - Which gain survives when the compute budget is matched?
49
+
50
+ ## References
51
+
52
+ [1] Sharma et al., Conceptual Captions, 2018.
53
+ [2] Hessel et al., CLIPScore, 2021.
54
+ [3] Li et al., BLIP-2, 2023.
training_args.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "optimizer": "adamw",
3
+ "scheduler": "cosine",
4
+ "learning_rate": 0.0001,
5
+ "batch_size": 16,
6
+ "epochs": 10,
7
+ "seed": 3407,
8
+ "status": "default recipe; not a completed run"
9
+ }