Upload TextGenerationReport

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

benchmark_report.json

@@ -0,0 +1,193 @@
+{
+    "prefill": {
+        "memory": {
+            "unit": "MB",
+            "max_ram": 3470.56128,
+            "max_vram": 12769.558528,
+            "max_reserved": 12276.727808,
+            "max_allocated": 10493.853184
+        },
+        "latency": {
+            "unit": "s",
+            "mean": 0.3322811594317036,
+            "stdev": 0.004201918534069472,
+            "values": [
+                0.3548702697753906,
+                0.3352504272460938,
+                0.3326003112792969,
+                0.33182821655273437,
+                0.3319469299316406,
+                0.33254296875,
+                0.3314892883300781,
+                0.33137869262695313,
+                0.33092608642578125,
+                0.3313489990234375,
+                0.3318661193847656,
+                0.3311390686035156,
+                0.33138790893554687,
+                0.3314851989746094,
+                0.33103564453125,
+                0.3311247253417969,
+                0.3315466003417969,
+                0.33106533813476563,
+                0.3314124755859375,
+                0.331188232421875,
+                0.33160498046875,
+                0.3317739562988281,
+                0.3311513671875,
+                0.33090765380859377,
+                0.33081649780273437,
+                0.33114727783203124,
+                0.33095269775390623,
+                0.3309649963378906,
+                0.3314667663574219,
+                0.33160498046875,
+                0.33089126586914064
+            ]
+        },
+        "throughput": {
+            "unit": "tokens/s",
+            "value": 722.2798921565973
+        },
+        "energy": null,
+        "efficiency": null
+    },
+    "decode": {
+        "memory": {
+            "unit": "MB",
+            "max_ram": 3471.822848,
+            "max_vram": 13524.533248,
+            "max_reserved": 13031.702528,
+            "max_allocated": 10594.238464
+        },
+        "latency": {
+            "unit": "s",
+            "mean": 27.10930010986329,
+            "stdev": 0,
+            "values": [
+                27.10930010986329
+            ]
+        },
+        "throughput": {
+            "unit": "tokens/s",
+            "value": 54.77824930860927
+        },
+        "energy": null,
+        "efficiency": null
+    },
+    "per_token": {
+        "memory": null,
+        "latency": {
+            "unit": "s",
+            "mean": 0.27383131424104334,
+            "stdev": 0.003865402878377042,
+            "values": [
+                0.28423577880859374,
+                0.2813306884765625,
+                0.28021041870117186,
+                0.2805688171386719,
+                0.2805350341796875,
+                0.280511474609375,
+                0.28056781005859377,
+                0.28048178100585935,
+                0.2796636047363281,
+                0.279478271484375,
+                0.2780057678222656,
+                0.2784327697753906,
+                0.27773236083984376,
+                0.2771885986328125,
+                0.2785331115722656,
+                0.27619430541992185,
+                0.27745895385742186,
+                0.27661618041992186,
+                0.2762188720703125,
+                0.27671243286132813,
+                0.27565362548828126,
+                0.2763612060546875,
+                0.27662130737304685,
+                0.2756331481933594,
+                0.27525225830078126,
+                0.2755604248046875,
+                0.27471261596679686,
+                0.2745743103027344,
+                0.27438796997070314,
+                0.27482009887695313,
+                0.2740787048339844,
+                0.2747821960449219,
+                0.2742958068847656,
+                0.27405517578125,
+                0.27351553344726565,
+                0.27380224609375,
+                0.2732697448730469,
+                0.273517578125,
+                0.27277618408203125,
+                0.2734888916015625,
+                0.27336294555664065,
+                0.27980593872070314,
+                0.27267788696289064,
+                0.27327078247070313,
+                0.2725263366699219,
+                0.2733834228515625,
+                0.2763018493652344,
+                0.28083709716796873,
+                0.2731366271972656,
+                0.28008447265625,
+                0.27196417236328124,
+                0.27271783447265624,
+                0.27304241943359375,
+                0.27160986328125,
+                0.2719815673828125,
+                0.2721392517089844,
+                0.2742210693359375,
+                0.27142962646484375,
+                0.2707630004882812,
+                0.27627825927734373,
+                0.2755911560058594,
+                0.27026943969726563,
+                0.27018649291992186,
+                0.2707906494140625,
+                0.2697359313964844,
+                0.26930584716796874,
+                0.27066574096679685,
+                0.26956494140625,
+                0.2701271057128906,
+                0.2692567138671875,
+                0.2692587585449219,
+                0.2694625244140625,
+                0.2702479248046875,
+                0.27053466796875,
+                0.27034725952148436,
+                0.2764933166503906,
+                0.27599053955078123,
+                0.2792693786621094,
+                0.2697052307128906,
+                0.2699970703125,
+                0.2686484375,
+                0.2691286926269531,
+                0.2699591674804688,
+                0.2695352783203125,
+                0.2692228698730469,
+                0.2694993896484375,
+                0.26948300170898437,
+                0.26922906494140625,
+                0.26847027587890626,
+                0.27224267578125,
+                0.2729758605957031,
+                0.27673394775390625,
+                0.2686033935546875,
+                0.26899252319335937,
+                0.269348876953125,
+                0.2679552001953125,
+                0.2716129150390625,
+                0.2688880615234375,
+                0.26859930419921874
+            ]
+        },
+        "throughput": {
+            "unit": "tokens/s",
+            "value": 54.77824930860927
+        },
+        "energy": null,
+        "efficiency": null
+    }
+}