Add files using upload-large-folder tool

.cache/pip/http-v2/0/7/3/c/3/073c319374cb73c3a72a13b4dc11439818d60565b30816a95031773f.body

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: av
+Version: 16.0.0
+Summary: Pythonic bindings for FFmpeg's libraries.
+Author-email: WyattBlue <wyattblue@auto-editor.com>, Jeremy Lainé <jeremy.laine@m4x.org>
+License-Expression: BSD-3-Clause
+Project-URL: Bug Tracker, https://github.com/PyAV-Org/PyAV/discussions/new?category=4-bugs
+Project-URL: Source Code, https://github.com/PyAV-Org/PyAV
+Project-URL: homepage, https://pyav.basswood-io.com
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: Unix
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Programming Language :: Cython
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Multimedia :: Sound/Audio
+Classifier: Topic :: Multimedia :: Sound/Audio :: Conversion
+Classifier: Topic :: Multimedia :: Video
+Classifier: Topic :: Multimedia :: Video :: Conversion
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+License-File: AUTHORS.py
+License-File: AUTHORS.rst
+Dynamic: license-file
+
+PyAV
+====
+
+PyAV is a Pythonic binding for the [FFmpeg][ffmpeg] libraries. We aim to provide all of the power and control of the underlying library, but manage the gritty details as much as possible.
+
+---
+
+[![GitHub Test Status][github-tests-badge]][github-tests] [![Documentation][docs-badge]][docs] [![Python Package Index][pypi-badge]][pypi] [![Conda Forge][conda-badge]][conda]
+
+PyAV is for direct and precise access to your media via containers, streams, packets, codecs, and frames. It exposes a few transformations of that data, and helps you get your data to/from other packages (e.g. Numpy and Pillow).
+
+This power does come with some responsibility as working with media is horrendously complicated and PyAV can't abstract it away or make all the best decisions for you. If the `ffmpeg` command does the job without you bending over backwards, PyAV is likely going to be more of a hindrance than a help.
+
+But where you can't work without it, PyAV is a critical tool.
+
+
+Installation
+------------
+
+Binary wheels are provided on [PyPI][pypi] for Linux, MacOS and Windows linked against the latest stable version of ffmpeg. You can install these wheels by running:
+
+```bash
+pip install av
+```
+
+Another way of installing PyAV is via [conda-forge][conda-forge]:
+
+```bash
+conda install av -c conda-forge
+```
+
+See the [Conda install][conda-install] docs to get started with (mini)Conda.
+
+
+Alternative installation methods
+--------------------------------
+
+Due to the complexity of the dependencies, PyAV is not always the easiest Python package to install from source. If you want to use your existing ffmpeg (must be the correct major version), the source version of PyAV is on [PyPI][pypi]:
+
+> [!WARNING]
+> You must be in a posix env, and have the correct version of ffmpeg installed on your system.
+
+```bash
+pip install av --no-binary av
+```
+
+
+Installing From Source
+----------------------
+
+Here's how to build PyAV from source. You must use [MSYS2](https://www.msys2.org/) when using Windows.
+
+```bash
+git clone https://github.com/PyAV-Org/PyAV.git
+cd PyAV
+source scripts/activate.sh
+
+# Build ffmpeg from source. You can skip this step
+# if ffmpeg is already installed.
+./scripts/build-deps
+
+# Build PyAV
+make
+
+# Testing
+make test
+
+# Install globally
+deactivate
+pip install .
+```
+
+---
+
+Have fun, [read the docs][docs], [come chat with us][discuss], and good luck!
+
+
+
+[conda-badge]: https://img.shields.io/conda/vn/conda-forge/av.svg?colorB=CCB39A
+[conda]: https://anaconda.org/conda-forge/av
+[docs-badge]: https://img.shields.io/badge/docs-on%20pyav.basswood--io.com-blue.svg
+[docs]: https://pyav.basswood-io.com
+[pypi-badge]: https://img.shields.io/pypi/v/av.svg?colorB=CCB39A
+[pypi]: https://pypi.org/project/av
+[discuss]: https://github.com/PyAV-Org/PyAV/discussions
+
+[github-tests-badge]: https://github.com/PyAV-Org/PyAV/workflows/tests/badge.svg
+[github-tests]: https://github.com/PyAV-Org/PyAV/actions?workflow=tests
+[github]: https://github.com/PyAV-Org/PyAV
+
+[ffmpeg]: https://ffmpeg.org/
+[conda-forge]: https://conda-forge.github.io/
+[conda-install]: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html

.cache/pip/http-v2/0/e/2/b/6/0e2b668f82a48b560e0865ba3099ebc225eb59d26d4badf227df001a.body

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: shtab
+Version: 1.8.0
+Summary: Automagic shell tab completion for Python CLI applications
+Author-email: Casper da Costa-Luis <casper.dcl@physics.org>
+Maintainer-email: Iterative <support@iterative.ai>
+License-Expression: Apache-2.0
+Project-URL: documentation, https://docs.iterative.ai/shtab
+Project-URL: repository, https://github.com/iterative/shtab
+Project-URL: changelog, https://github.com/iterative/shtab/releases
+Keywords: tab,complete,completion,shell,bash,zsh,argparse
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Environment :: MacOS X
+Classifier: Environment :: Other Environment
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: Intended Audience :: Other Audience
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: POSIX :: BSD
+Classifier: Operating System :: POSIX :: BSD :: FreeBSD
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Operating System :: POSIX :: SunOS/Solaris
+Classifier: Operating System :: Unix
+Classifier: Programming Language :: Other Scripting Engines
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: Implementation
+Classifier: Programming Language :: Python :: Implementation :: IronPython
+Classifier: Programming Language :: Python :: Implementation :: PyPy
+Classifier: Programming Language :: Unix Shell
+Classifier: Topic :: Desktop Environment
+Classifier: Topic :: Education :: Computer Aided Instruction (CAI)
+Classifier: Topic :: Education :: Testing
+Classifier: Topic :: Office/Business
+Classifier: Topic :: Other/Nonlisted Topic
+Classifier: Topic :: Software Development
+Classifier: Topic :: Software Development :: Build Tools
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Software Development :: Pre-processors
+Classifier: Topic :: Software Development :: User Interfaces
+Classifier: Topic :: System
+Classifier: Topic :: System :: Installation/Setup
+Classifier: Topic :: System :: Shells
+Classifier: Topic :: System :: System Shells
+Classifier: Topic :: Terminals
+Classifier: Topic :: Utilities
+Requires-Python: >=3.9
+Description-Content-Type: text/x-rst
+License-File: LICENCE
+Provides-Extra: dev
+Requires-Dist: pytest>=6; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: pytest-timeout; extra == "dev"
+Dynamic: license-file
+
+|Logo|
+
+shtab
+=====
+
+|PyPI-Downloads| |Tests| |Coverage| |PyPI| |Conda|
+
+- What: Automatically generate shell tab completion scripts for Python CLI apps
+- Why: Speed & correctness. Alternatives like
+  `argcomplete <https://pypi.org/project/argcomplete>`_ and
+  `pyzshcomplete <https://pypi.org/project/pyzshcomplete>`_ are slow and have
+  side-effects
+- How: ``shtab`` processes an ``argparse.ArgumentParser`` object to generate a
+  tab completion script for your shell
+
+Features
+--------
+
+- Outputs tab completion scripts for
+
+  - ``bash``
+  - ``zsh``
+  - ``tcsh``
+
+- Supports
+
+  - `argparse <https://docs.python.org/library/argparse>`_
+  - `docopt <https://pypi.org/project/docopt>`_ (via `argopt <https://pypi.org/project/argopt>`_)
+
+- Supports arguments, options and subparsers
+- Supports choices (e.g. ``--say={hello,goodbye}``)
+- Supports file and directory path completion
+- Supports custom path completion (e.g. ``--file={*.txt}``)
+
+------------------------------------------
+
+.. contents:: Table of Contents
+   :backlinks: top
+
+Installation
+------------
+
+Choose one of:
+
+- ``pip install shtab``, or
+- ``conda install -c conda-forge shtab``
+
+See `operating system-specific instructions in the docs <https://docs.iterative.ai/shtab/#installation>`_.
+
+Usage
+-----
+
+There are two ways of using ``shtab``:
+
+- `CLI Usage <https://docs.iterative.ai/shtab/use/#cli-usage>`_: ``shtab``'s own CLI interface for external applications
+
+  - may not require any code modifications whatsoever
+  - end-users execute ``shtab your_cli_app.your_parser_object``
+
+- `Library Usage <https://docs.iterative.ai/shtab/use/#library-usage>`_: as a library integrated into your CLI application
+
+  - adds a couple of lines to your application
+  - argument mode: end-users execute ``your_cli_app --print-completion {bash,zsh,tcsh}``
+  - subparser mode: end-users execute ``your_cli_app completion {bash,zsh,tcsh}``
+
+Examples
+--------
+
+See `the docs for usage examples <https://docs.iterative.ai/shtab/use/#main.py>`_.
+
+FAQs
+----
+
+Not working? Check out `frequently asked questions <https://docs.iterative.ai/shtab/#faqs>`_.
+
+Alternatives
+------------
+
+- `argcomplete <https://pypi.org/project/argcomplete>`_
+
+  - executes the underlying script *every* time ``<TAB>`` is pressed (slow and
+    has side-effects)
+
+- `pyzshcomplete <https://pypi.org/project/pyzshcomplete>`_
+
+  - executes the underlying script *every* time ``<TAB>`` is pressed (slow and
+    has side-effects)
+  - only provides ``zsh`` completion
+
+- `click <https://pypi.org/project/click>`_
+
+  - different framework completely replacing the builtin ``argparse``
+  - solves multiple problems (rather than POSIX-style "do one thing well")
+
+Contributions
+-------------
+
+Please do open `issues <https://github.com/iterative/shtab/issues>`_ & `pull requests <https://github.com/iterative/shtab/pulls>`_! Some ideas:
+
+- support ``fish`` (`#174 <https://github.com/iterative/shtab/pull/174>`_)
+- support ``powershell``
+
+See
+`CONTRIBUTING.md <https://github.com/iterative/shtab/tree/main/CONTRIBUTING.md>`_
+for more guidance.
+
+|Hits|
+
+.. |Logo| image:: https://github.com/iterative/shtab/raw/main/meta/logo.png
+.. |Tests| image:: https://img.shields.io/github/actions/workflow/status/iterative/shtab/test.yml?logo=github&label=tests
+   :target: https://github.com/iterative/shtab/actions
+   :alt: Tests
+.. |Coverage| image:: https://codecov.io/gh/iterative/shtab/branch/main/graph/badge.svg
+   :target: https://codecov.io/gh/iterative/shtab
+   :alt: Coverage
+.. |Conda| image:: https://img.shields.io/conda/v/conda-forge/shtab.svg?label=conda&logo=conda-forge
+   :target: https://anaconda.org/conda-forge/shtab
+   :alt: conda-forge
+.. |PyPI| image:: https://img.shields.io/pypi/v/shtab.svg?label=pip&logo=PyPI&logoColor=white
+   :target: https://pypi.org/project/shtab
+   :alt: PyPI
+.. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/shtab.svg?label=pypi%20downloads&logo=PyPI&logoColor=white
+   :target: https://pepy.tech/project/shtab
+   :alt: Downloads
+.. |Hits| image:: https://cgi.cdcl.ml/hits?q=shtab&style=social&r=https://github.com/iterative/shtab&a=hidden
+   :target: https://cgi.cdcl.ml/hits?q=shtab&a=plot&r=https://github.com/iterative/shtab&style=social
+   :alt: Hits

.cache/pip/http-v2/3/1/2/8/c/3128c2672ba9ca8952d2c175538dc592d0105b1a9f9a771148c3a28f.body

@@ -0,0 +1,17 @@
+Metadata-Version: 2.1
+Name: protobuf
+Author: protobuf@googlegroups.com
+Author-email: protobuf@googlegroups.com
+Home-page: https://developers.google.com/protocol-buffers/
+License: 3-Clause BSD License
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.9
+Version: 6.33.5
+
+UNKNOWN

.cache/pip/http-v2/3/8/e/d/8/38ed89a6c68dd747f9bfdf6c4269891176f046dfcecb4f35f2564158.body

@@ -0,0 +1,394 @@
+Metadata-Version: 2.4
+Name: einops
+Version: 0.8.2
+Summary: A new flavour of deep learning operations
+Project-URL: Homepage, https://github.com/arogozhnikov/einops
+Author: Alex Rogozhnikov
+License: MIT
+License-File: LICENSE
+Keywords: deep learning,einops,machine learning,neural networks,scientific computations,tensor manipulation
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+
+<!--
+<a href='http://arogozhnikov.github.io/images/einops/einops_video.mp4' >
+<div align="center">
+  <img src="http://arogozhnikov.github.io/images/einops/einops_video.gif" alt="einops package examples" />
+  <br>
+  <small><a href='http://arogozhnikov.github.io/images/einops/einops_video.mp4'>This video in high quality (mp4)</a></small>
+  <br><br>
+</div>
+</a>
+-->
+
+<!-- this link magically rendered as video on github readme, unfortunately not in docs -->
+
+https://user-images.githubusercontent.com/6318811/177030658-66f0eb5d-e136-44d8-99c9-86ae298ead5b.mp4
+
+
+
+
+# einops 
+[![Run tests](https://github.com/arogozhnikov/einops/actions/workflows/run_tests.yml/badge.svg)](https://github.com/arogozhnikov/einops/actions/workflows/run_tests.yml)
+[![PyPI version](https://badge.fury.io/py/einops.svg)](https://badge.fury.io/py/einops)
+[![Documentation](https://img.shields.io/badge/documentation-link-blue.svg)](https://einops.rocks/)
+![Supported python versions](https://raw.githubusercontent.com/arogozhnikov/einops/main/docs/resources/python_badge.svg)
+
+
+Flexible and powerful tensor operations for readable and reliable code. <br />
+Supports numpy, pytorch, tensorflow, jax, and [others](#supported-frameworks).
+
+## Recent updates:
+
+- 0.8.0: tinygrad backend added, small fixes
+- 0.7.0: no-hassle `torch.compile`, support of [array api standard](https://data-apis.org/array-api/latest/API_specification/index.html) and more
+- 10'000🎉: github reports that more than 10k project use einops
+- einops 0.6.1: paddle backend added
+- einops 0.6 introduces [packing and unpacking](https://github.com/arogozhnikov/einops/blob/main/docs/4-pack-and-unpack.ipynb)
+- einops 0.5: einsum is now a part of einops
+- [Einops paper](https://openreview.net/pdf?id=oapKSVM2bcj) is accepted for oral presentation at ICLR 2022 (yes, it worth reading).
+  Talk recordings are [available](https://iclr.cc/virtual/2022/oral/6603)
+
+
+<details markdown="1">
+<summary>Previous updates</summary>
+- flax and oneflow backend added
+- torch.jit.script is supported for pytorch layers
+- powerful EinMix added to einops. [Einmix tutorial notebook](https://github.com/arogozhnikov/einops/blob/main/docs/3-einmix-layer.ipynb) 
+</details>
+
+<!--<div align="center">
+  <img src="http://arogozhnikov.github.io/images/einops/einops_logo_350x350.png" 
+  alt="einops package logo" width="250" height="250" />
+  <br><br>
+</div> -->
+
+
+## Tweets 
+
+> In case you need convincing arguments for setting aside time to learn about einsum and einops...
+[Tim Rocktäschel](https://twitter.com/_rockt/status/1230818967205425152)
+
+> Writing better code with PyTorch and einops 👌
+[Andrej Karpathy](https://twitter.com/karpathy/status/1290826075916779520)
+
+> Slowly but surely, einops is seeping in to every nook and cranny of my code. If you find yourself shuffling around bazillion dimensional tensors, this might change your life
+[Nasim Rahaman](https://twitter.com/nasim_rahaman/status/1216022614755463169)
+
+[More testimonials](https://einops.rocks/pages/testimonials/)
+
+
+## Contents
+
+- [Installation](#Installation)
+- [Documentation](https://einops.rocks/)
+- [Tutorial](#Tutorials)
+- [API micro-reference](#API)
+- [Why use einops](#Why-use-einops-notation)
+- [Supported frameworks](#Supported-frameworks)
+- [Citing](#Citing)
+- [Repository](https://github.com/arogozhnikov/einops) and [discussions](https://github.com/arogozhnikov/einops/discussions)
+
+## Installation  <a name="Installation"></a>
+
+Plain and simple:
+```bash
+pip install einops
+```
+
+(`uv pip install einops` works as well)
+
+## Tutorials <a name="Tutorials"></a>
+
+Tutorials are the most convenient way to see `einops` in action
+
+- part 1: [einops fundamentals](https://github.com/arogozhnikov/einops/blob/main/docs/1-einops-basics.ipynb)
+- part 2: [einops for deep learning](https://github.com/arogozhnikov/einops/blob/main/docs/2-einops-for-deep-learning.ipynb)
+- part 3: [packing and unpacking](https://github.com/arogozhnikov/einops/blob/main/docs/4-pack-and-unpack.ipynb)
+- part 4: [improve pytorch code with einops](http://einops.rocks/pytorch-examples.html)
+
+Kapil Sachdeva recorded a small [intro to einops](https://www.youtube.com/watch?v=xGy75Pjsqzo).
+
+## API <a name="API"></a>
+
+`einops` has a minimalistic yet powerful API.
+
+Three core operations provided ([einops tutorial](https://github.com/arogozhnikov/einops/blob/main/docs/)
+shows those cover stacking, reshape, transposition, squeeze/unsqueeze, repeat, tile, concatenate, view and numerous reductions)
+
+```python
+from einops import rearrange, reduce, repeat
+# rearrange elements according to the pattern
+output_tensor = rearrange(input_tensor, 't b c -> b c t')
+# combine rearrangement and reduction
+output_tensor = reduce(input_tensor, 'b c (h h2) (w w2) -> b h w c', 'mean', h2=2, w2=2)
+# copy along a new axis
+output_tensor = repeat(input_tensor, 'h w -> h w c', c=3)
+```
+
+Later additions to the family are `pack` and `unpack` functions (better than stack/split/concatenate):
+
+```python
+from einops import pack, unpack
+# pack and unpack allow reversibly 'packing' multiple tensors into one.
+# Packed tensors may be of different dimensionality:
+packed,  ps = pack([class_token_bc, image_tokens_bhwc, text_tokens_btc], 'b * c')
+class_emb_bc, image_emb_bhwc, text_emb_btc = unpack(transformer(packed), ps, 'b * c')
+```
+
+Finally, einops provides einsum with a support of multi-lettered names:
+
+```python
+from einops import einsum, pack, unpack
+# einsum is like ... einsum, generic and flexible dot-product
+# but 1) axes can be multi-lettered  2) pattern goes last 3) works with multiple frameworks
+C = einsum(A, B, 'b t1 head c, b t2 head c -> b head t1 t2')
+```
+
+### EinMix
+
+`EinMix` is a generic linear layer, perfect for MLP Mixers and similar architectures.
+
+### Layers
+
+Einops provides layers (`einops` keeps a separate version for each framework) that reflect corresponding functions
+
+```python
+from einops.layers.torch      import Rearrange, Reduce
+from einops.layers.tensorflow import Rearrange, Reduce
+from einops.layers.flax       import Rearrange, Reduce
+from einops.layers.paddle     import Rearrange, Reduce
+```
+
+<details markdown="1">
+<summary>Example of using layers within a pytorch model</summary>
+Example given for pytorch, but code in other frameworks is almost identical
+
+```python 
+from torch.nn import Sequential, Conv2d, MaxPool2d, Linear, ReLU
+from einops.layers.torch import Rearrange
+
+model = Sequential(
+    ...,
+    Conv2d(6, 16, kernel_size=5),
+    MaxPool2d(kernel_size=2),
+    # flattening without need to write forward
+    Rearrange('b c h w -> b (c h w)'),
+    Linear(16*5*5, 120),
+    ReLU(),
+    Linear(120, 10),
+)
+```
+
+No more flatten needed!
+
+Additionally, torch layers as those are script-able and compile-able.
+Operations [are torch.compile-able](https://github.com/arogozhnikov/einops/wiki/Using-torch.compile-with-einops),
+ but not script-able due to limitations of torch.jit.script.
+</details>
+
+
+
+
+## Naming <a name="Naming"></a>
+
+`einops` stands for Einstein-Inspired Notation for operations 
+(though "Einstein operations" is more attractive and easier to remember).
+
+Notation was loosely inspired by Einstein summation (in particular by `numpy.einsum` operation).
+
+## Why use `einops` notation?! <a name="Why-use-einops-notation"></a>
+
+
+### Semantic information (being verbose in expectations)
+
+```python
+y = x.view(x.shape[0], -1)
+y = rearrange(x, 'b c h w -> b (c h w)')
+```
+While these two lines are doing the same job in *some* context,
+the second one provides information about the input and output.
+In other words, `einops` focuses on interface: *what is the input and output*, not *how* the output is computed.
+
+The next operation looks similar:
+
+```python
+y = rearrange(x, 'time c h w -> time (c h w)')
+```
+but it gives the reader a hint:
+this is not an independent batch of images we are processing,
+but rather a sequence (video).
+
+Semantic information makes the code easier to read and maintain.
+
+### Convenient checks
+
+Reconsider the same example:
+
+```python
+y = x.view(x.shape[0], -1) # x: (batch, 256, 19, 19)
+y = rearrange(x, 'b c h w -> b (c h w)')
+```
+The second line checks that the input has four dimensions,
+but you can also specify particular dimensions.
+That's opposed to just writing comments about shapes since comments don't prevent mistakes,
+not tested, and without code review tend to be outdated
+```python
+y = x.view(x.shape[0], -1) # x: (batch, 256, 19, 19)
+y = rearrange(x, 'b c h w -> b (c h w)', c=256, h=19, w=19)
+```
+
+### Result is strictly determined
+
+Below we have at least two ways to define the depth-to-space operation
+```python
+# depth-to-space
+rearrange(x, 'b c (h h2) (w w2) -> b (c h2 w2) h w', h2=2, w2=2)
+rearrange(x, 'b c (h h2) (w w2) -> b (h2 w2 c) h w', h2=2, w2=2)
+```
+There are at least four more ways to do it. Which one is used by the framework?
+
+These details are ignored, since *usually* it makes no difference,
+but it can make a big difference (e.g. if you use grouped convolutions in the next stage),
+and you'd like to specify this in your code.
+
+
+### Uniformity
+
+```python
+reduce(x, 'b c (x dx) -> b c x', 'max', dx=2)
+reduce(x, 'b c (x dx) (y dy) -> b c x y', 'max', dx=2, dy=3)
+reduce(x, 'b c (x dx) (y dy) (z dz) -> b c x y z', 'max', dx=2, dy=3, dz=4)
+```
+These examples demonstrated that we don't use separate operations for 1d/2d/3d pooling,
+those are all defined in a uniform way.
+
+Space-to-depth and depth-to space are defined in many frameworks but how about width-to-height? Here you go:
+
+```python
+rearrange(x, 'b c h (w w2) -> b c (h w2) w', w2=2)
+```
+
+
+### Framework independent behavior
+
+Even simple functions are defined differently by different frameworks
+
+```python
+y = x.flatten() # or flatten(x)
+```
+
+Suppose `x`'s shape was `(3, 4, 5)`, then `y` has shape ...
+
+- numpy, pytorch, cupy, chainer, jax: `(60,)`
+- keras, tensorflow.layers, gluon: `(3, 20)`
+
+`einops` works the same way in all frameworks.
+
+
+### Independence of framework terminology
+
+Example: `tile` vs `repeat` causes lots of confusion. To copy image along width:
+```python
+np.tile(image, (1, 2))    # in numpy
+image.repeat(1, 2)        # pytorch's repeat ~ numpy's tile
+```
+
+With einops you don't need to decipher which axis was repeated:
+```python
+repeat(image, 'h w -> h (tile w)', tile=2)  # in numpy
+repeat(image, 'h w -> h (tile w)', tile=2)  # in pytorch
+repeat(image, 'h w -> h (tile w)', tile=2)  # in tf
+repeat(image, 'h w -> h (tile w)', tile=2)  # in jax
+repeat(image, 'h w -> h (tile w)', tile=2)  # in cupy
+... (etc.)
+```
+
+[Testimonials](https://einops.rocks/pages/testimonials/) provide users' perspective on the same question.
+
+
+## Supported frameworks <a name="Supported-frameworks"></a>
+
+Einops works with ...
+
+- [numpy](http://www.numpy.org/)
+- [pytorch](https://pytorch.org/)
+- [tensorflow](https://www.tensorflow.org/)
+- [jax](https://github.com/google/jax)
+- [cupy](https://github.com/cupy/cupy)
+- [flax](https://github.com/google/flax) (community)
+- [paddle](https://github.com/PaddlePaddle/Paddle) (community)
+- [oneflow](https://github.com/Oneflow-Inc/oneflow) (community)
+- [tinygrad](https://github.com/tinygrad/tinygrad) (community)
+- [pytensor](https://github.com/pymc-devs/pytensor) (community)
+
+
+
+```python
+from einops import rearrange  
+=> from einops.array_api import rearrange
+```
+
+But actually it is even better: einops can be used with *any* framework that supports
+[Python array API standard](https://data-apis.org/array-api/latest/API_specification/index.html),
+to name a few:
+
+- numpy >= 2.0
+- [MLX](https://github.com/ml-explore/mlx)  # yes, einops works with apple's framework
+- [pydata/sparse](https://github.com/pydata/sparse) >= 0.15 # and works with sparse tensors
+- [cubed](https://github.com/cubed-dev/cubed) # and with distributed tensors too
+- [quantco/ndonnx](https://github.com/Quantco/ndonnx)
+- jax
+- cupy
+- dask is supported via [array-api-compat](https://github.com/data-apis/array-api-compat)
+
+
+## Development
+
+Devcontainer is provided, this environment can be used locally, or on your server,
+or within github codespaces. 
+To start with devcontainers in vs code, clone repo, and click 'Reopen in Devcontainer'. 
+
+Starting from einops 0.8.1, einops distributes tests as a part of package.
+
+```bash
+# pip install einops pytest
+python -m einops.tests.run_tests numpy pytorch jax --pip-install
+```
+
+`numpy pytorch jax` is an _example_, any subset of testable frameworks can be provided.
+Every framework is tested against numpy, so it is a requirement for tests.
+
+Specifying `--pip-install` will install requirements in current virtualenv,
+and should be omitted if dependencies are installed locally.
+
+To build/test docs:
+
+```bash
+hatch run docs:serve  # Serving on http://localhost:8000/
+```
+
+
+## Citing einops <a name="Citing"></a>
+
+Please use the following bibtex record
+
+```text
+@inproceedings{
+    rogozhnikov2022einops,
+    title={Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation},
+    author={Alex Rogozhnikov},
+    booktitle={International Conference on Learning Representations},
+    year={2022},
+    url={https://openreview.net/forum?id=oapKSVM2bcj}
+}
+```
+
+
+## Supported python versions
+
+`einops` works with python 3.9 or later.

.cache/pip/http-v2/3/c/e/7/7/3ce770df07ea321ab056bed31a67391a50e82aea9460d349fd057276.body

@@ -0,0 +1,107 @@
+Metadata-Version: 2.4
+Name: packaging
+Version: 26.0
+Summary: Core utilities for Python packages
+Author-email: Donald Stufft <donald@stufft.io>
+Requires-Python: >=3.8
+Description-Content-Type: text/x-rst
+License-Expression: Apache-2.0 OR BSD-2-Clause
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Programming Language :: Python :: Implementation :: PyPy
+Classifier: Typing :: Typed
+License-File: LICENSE
+License-File: LICENSE.APACHE
+License-File: LICENSE.BSD
+Project-URL: Documentation, https://packaging.pypa.io/
+Project-URL: Source, https://github.com/pypa/packaging
+
+packaging
+=========
+
+.. start-intro
+
+Reusable core utilities for various Python Packaging
+`interoperability specifications <https://packaging.python.org/specifications/>`_.
+
+This library provides utilities that implement the interoperability
+specifications which have clearly one correct behaviour (eg: :pep:`440`)
+or benefit greatly from having a single shared implementation (eg: :pep:`425`).
+
+.. end-intro
+
+The ``packaging`` project includes the following: version handling, specifiers,
+markers, requirements, tags, metadata, lockfiles, utilities.
+
+Documentation
+-------------
+
+The `documentation`_ provides information and the API for the following:
+
+- Version Handling
+- Specifiers
+- Markers
+- Requirements
+- Tags
+- Metadata
+- Lockfiles
+- Utilities
+
+Installation
+------------
+
+Use ``pip`` to install these utilities::
+
+    pip install packaging
+
+The ``packaging`` library uses calendar-based versioning (``YY.N``).
+
+Discussion
+----------
+
+If you run into bugs, you can file them in our `issue tracker`_.
+
+You can also join ``#pypa`` on Freenode to ask questions or get involved.
+
+
+.. _`documentation`: https://packaging.pypa.io/
+.. _`issue tracker`: https://github.com/pypa/packaging/issues
+
+
+Code of Conduct
+---------------
+
+Everyone interacting in the packaging project's codebases, issue trackers, chat
+rooms, and mailing lists is expected to follow the `PSF Code of Conduct`_.
+
+.. _PSF Code of Conduct: https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md
+
+Contributing
+------------
+
+The ``CONTRIBUTING.rst`` file outlines how to contribute to this project as
+well as how to report a potential security issue. The documentation for this
+project also covers information about `project development`_ and `security`_.
+
+.. _`project development`: https://packaging.pypa.io/en/latest/development/
+.. _`security`: https://packaging.pypa.io/en/latest/security/
+
+Project History
+---------------
+
+Please review the ``CHANGELOG.rst`` file or the `Changelog documentation`_ for
+recent changes and project history.
+
+.. _`Changelog documentation`: https://packaging.pypa.io/en/latest/changelog/
+

.cache/pip/http-v2/3/d/8/c/a/3d8cad774d82fa02cd6b57b61c7ae5f445e51121f6b8147871d094e2.body

@@ -0,0 +1,251 @@
+Metadata-Version: 2.4
+Name: sentencepiece
+Version: 0.2.1
+Summary: Unsupervised text tokenizer and detokenizer.
+Author-email: Taku Kudo <taku@google.com>
+Project-URL: Homepage, https://github.com/google/sentencepiece
+Classifier: Programming Language :: Python :: 3
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: Free Threading :: 2 - Beta
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Provides-Extra: testpaths
+Requires-Dist: test; extra == "testpaths"
+
+# SentencePiece Python Wrapper
+
+Python wrapper for SentencePiece. This API will offer the encoding, decoding and training of Sentencepiece.
+
+## Build and Install SentencePiece
+
+For Linux (x64/i686), macOS, and Windows(win32/x64/arm64) environment, you can simply use pip command to install SentencePiece python module.
+
+```
+% pip install sentencepiece
+```
+
+Before building SentencePiece from source on Linux, ensure that the following dependencies are installed.
+
+```
+% sudo apt update
+% sudo apt install -y cmake pkg-config libsentencepiece-dev
+```
+
+To build and install the Python wrapper from source, try the following commands to build and install wheel package.
+
+```
+% git clone https://github.com/google/sentencepiece.git
+% cd sentencepiece
+% mkdir build
+% cd build
+% cmake .. -DSPM_ENABLE_SHARED=OFF -DCMAKE_INSTALL_PREFIX=./root -DSPM_DISABLE_EMBEDDED_DATA=ON
+% make install
+% cd ../python
+% python setup.py bdist_wheel
+% pip install dist/sentencepiece*.whl
+```
+
+If you don’t have write permission to the global site-packages directory or don’t want to install into it, please try:
+
+```
+% python setup.py install --user
+```
+
+For Windows users who want to build from source, you can build and install the Python wrapper using Visual Studio. First, you need to install the `pwsh.exe` (Powershell 7). Use `winget install --id Microsoft.Powershell --source winget` to install directly. Then open the `Developer PowerShell for VS 2022`, and execute the following commands.
+
+```
+git clone https://github.com/google/sentencepiece.git
+cd sentencepiece
+mkdir build
+cd build
+cmake .. -DSPM_ENABLE_SHARED=OFF -DCMAKE_INSTALL_PREFIX=".\root" -DSPM_DISABLE_EMBEDDED_DATA=ON
+cmake --build . --config Release --target install
+cd ../python
+pip install wheel
+python setup.py bdist_wheel
+Get-ChildItem .\dist\sentencepiece*.whl | ForEach-Object { pip install $_.FullName }
+```
+
+## Usage
+
+See [this google colab page](https://github.com/google/sentencepiece/blob/master/python/sentencepiece_python_module_example.ipynb) to run sentencepiece interactively.
+
+### Segmentation
+
+```
+% python
+>>> import sentencepiece as spm
+>>> sp = spm.SentencePieceProcessor(model_file='test/test_model.model')
+
+>>> sp.encode('This is a test')
+[284, 47, 11, 4, 15, 400]
+
+>>> sp.encode(['This is a test', 'Hello world'], out_type=int)
+[[284, 47, 11, 4, 15, 400], [151, 88, 21, 887]]
+
+>>> sp.encode_as_ids(['This is a test', 'Hello world'])
+[[284, 47, 11, 4, 15, 400], [151, 88, 21, 887]]
+
+>>> sp.encode('This is a test', out_type=str)
+['▁This', '▁is', '▁a', '▁', 't', 'est']
+
+>>> sp.encode(['This is a test', 'Hello world'], out_type=str)
+[['▁This', '▁is', '▁a', '▁', 't', 'est'], ['▁He', 'll', 'o', '▁world']]
+
+>>> sp.encode_as_pieces(['This is a test', 'Hello world'])
+[['▁This', '▁is', '▁a', '▁', 't', 'est'], ['▁He', 'll', 'o', '▁world']]
+
+>>> proto = sp.encode('This is a test', out_type='immutable_proto')
+>>> for n in proto.pieces:
+...     print('piece="{}" surface="{}" id={} begin={} end={}'.format(n.piece, n.surface, n.id, n.begin, n.end))
+...
+piece="▁This" surface="This" id=284 begin=0 end=4
+piece="▁is" surface=" is" id=47 begin=4 end=7
+piece="▁a" surface=" a" id=11 begin=7 end=9
+piece="▁" surface=" " id=4 begin=9 end=10
+piece="t" surface="t" id=15 begin=10 end=11
+piece="est" surface="est" id=400 begin=11 end=14
+
+>>> [[x.id for x in proto.pieces], [x.piece for x in proto.pieces], [x.begin for x in proto.pieces], [x.end for x in proto.pieces]]
+[[284, 47, 11, 4, 15, 400], ['▁This', '▁is', '▁a', '▁', 't', 'est'], [0, 4, 7, 9, 10, 11], [4, 7, 9, 10, 11, 14]]
+
+>>> proto2 = sp.encode_as_immutable_proto('This is a test')
+>>> proto2 == proto
+True
+
+>>> for _ in range(10):
+...     sp.encode('This is a test', out_type=str, enable_sampling=True, alpha=0.1, nbest_size=-1)
+...
+['▁', 'This', '▁', 'is', '▁a', '▁', 't', 'e', 'st']
+['▁T', 'h', 'i', 's', '▁is', '▁a', '▁', 'te', 's', 't']
+['▁T', 'h', 'is', '▁', 'is', '▁', 'a', '▁', 't', 'est']
+['▁', 'This', '▁is', '▁', 'a', '▁', 't', 'e', 'st']
+['▁', 'This', '▁', 'is', '▁', 'a', '▁', 't', 'e', 's', 't']
+['▁This', '▁is', '▁a', '▁', 'te', 's', 't']
+['▁This', '▁is', '▁', 'a', '▁', 't', 'e', 'st']
+['▁', 'T', 'h', 'is', '▁', 'is', '▁', 'a', '▁', 'te', 'st']
+['▁', 'This', '▁', 'i', 's', '▁a', '▁', 't', 'e', 'st']
+['▁This', '▁', 'is', '▁a', '▁', 't', 'est']
+
+>> sp.nbest_encode('This is a test', nbest_size=5, out_type=str)
+[['▁This', '▁is', '▁a', '▁', 't', 'est'],
+['▁This', '▁is', '▁a', '▁', 'te', 'st'],
+['▁This', '▁is', '▁a', '▁', 'te', 's', 't'],
+['▁This', '▁is', '▁a', '▁', 't', 'e', 'st'],
+['▁This', '▁is', '▁a', '▁', 't', 'es', 't']]
+
+>>> sp.sample_encode_and_score('This is a test', num_samples=5, alpha=0.1, out_type=str, wor=True)
+[(['▁This', '▁', 'i', 's', '▁a', '▁', 'te', 's', 't'], -3.043105125427246),
+(['▁This', '▁', 'i', 's', '▁a', '▁', 'te', 'st'], -2.8475849628448486),
+(['▁', 'This', '▁is', '▁', 'a', '▁', 'te', 'st'], -3.043248176574707),
+(['▁', 'This', '▁is', '▁a', '▁', 't', 'e', 'st'], -2.87727689743042),
+(['▁', 'This', '▁', 'i', 's', '▁', 'a', '▁', 't', 'est'], -3.6284031867980957)]
+
+>>> sp.decode([284, 47, 11, 4, 15, 400])
+'This is a test'
+
+>>> sp.decode([[284, 47, 11, 4, 15, 400], [151, 88, 21, 887]])
+['This is a test', 'Hello world']
+
+>>> proto = sp.decode([284, 47, 11, 4, 15, 400], out_type='immutable_proto')
+>>> proto.text
+'This is a test'
+
+>>> sp.decode(['▁', 'This', '▁', 'is', '▁a', '▁', 't', 'e', 'st'])
+'This is a test'
+
+>>> sp.decode([['▁This', '▁is', '▁a', '▁', 't', 'est'], ['▁He', 'll', 'o', '▁world']])
+['This is a test', 'Hello world']
+
+>>> sp.get_piece_size()
+1000
+
+>>> sp.id_to_piece(2)
+'</s>'
+
+>>> sp.id_to_piece([2, 3, 4])
+['</s>', '\r', '▁']
+
+>>> sp.piece_to_id('<s>')
+1
+
+>>> sp.piece_to_id(['</s>', '\r', '▁'])
+[2, 3, 4]
+
+>>> len(sp)
+1000
+
+>>> sp['</s>']
+2
+```
+
+### Model Training
+
+Training is performed by passing parameters of [spm_train](https://github.com/google/sentencepiece#train-sentencepiece-model) to SentencePieceTrainer.train() function.
+
+```
+>>> import sentencepiece as spm
+>>> spm.SentencePieceTrainer.train(input='test/botchan.txt', model_prefix='m', vocab_size=1000, user_defined_symbols=['foo', 'bar'])
+sentencepiece_trainer.cc(73) LOG(INFO) Starts training with :
+trainer_spec {
+  input: test/botchan.txt
+  .. snip
+unigram_model_trainer.cc(500) LOG(INFO) EM sub_iter=1 size=1188 obj=10.2839 num_tokens=32182 num_tokens/piece=27.0892
+unigram_model_trainer.cc(500) LOG(INFO) EM sub_iter=0 size=1100 obj=10.4269 num_tokens=33001 num_tokens/piece=30.0009
+unigram_model_trainer.cc(500) LOG(INFO) EM sub_iter=1 size=1100 obj=10.4069 num_tokens=33002 num_tokens/piece=30.0018
+trainer_interface.cc(595) LOG(INFO) Saving model: m.model
+trainer_interface.cc(619) LOG(INFO) Saving vocabs: m.vocab
+>>>
+```
+
+### Training without local filesystem
+
+Sentencepiece trainer can receive any iterable object to feed training sentences. You can also pass a file object (instance with write() method) to emit the output model to any devices. These features are useful to run sentencepiece on environment that have limited access to the local file system (e.g., Google colab.)
+
+```
+import urllib.request
+import io
+import sentencepiece as spm
+
+# Loads model from URL as iterator and stores the model to BytesIO.
+model = io.BytesIO()
+with urllib.request.urlopen(
+    'https://raw.githubusercontent.com/google/sentencepiece/master/data/botchan.txt'
+) as response:
+  spm.SentencePieceTrainer.train(
+      sentence_iterator=response, model_writer=model, vocab_size=1000)
+
+# Serialize the model as file.
+# with open('out.model', 'wb') as f:
+#   f.write(model.getvalue())
+
+# Directly load the model from serialized model.
+sp = spm.SentencePieceProcessor(model_proto=model.getvalue())
+print(sp.encode('this is test'))
+```
+
+### Free Threading support
+Experimental support for no-GIL/Free-Threading has been introduced since v0.2.1. For more details, please refer to [this page](https://py-free-threading.github.io.).
+This operates similarly to how [NumPy](https://numpy.org/devdocs/reference/thread_safety.html#free-threaded-python) handles it.
+
+The C++ library's const and static methods, e.g., encode(), decode() and train(), are designed to work in a non-GIL environment.
+However, non-const methods, e.g., load(), may have potential data race issues, so please ensure you implement appropriate locks beforehand.
+
+While this limitation might be removed in the future, please note that it's not a simple fix, as it would require additional shared locks in C++.

.cache/pip/http-v2/3/d/a/4/6/3da464de30c7468304779380c74eab34e7778fd6716e1e47bbe62e37.body

@@ -0,0 +1,265 @@
+Metadata-Version: 2.4
+Name: peft
+Version: 0.18.1
+Summary: Parameter-Efficient Fine-Tuning (PEFT)
+Home-page: https://github.com/huggingface/peft
+Author: The HuggingFace team
+Author-email: benjamin@huggingface.co
+License: Apache
+Keywords: deep learning
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10.0
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.17
+Requires-Dist: packaging>=20.0
+Requires-Dist: psutil
+Requires-Dist: pyyaml
+Requires-Dist: torch>=1.13.0
+Requires-Dist: transformers
+Requires-Dist: tqdm
+Requires-Dist: accelerate>=0.21.0
+Requires-Dist: safetensors
+Requires-Dist: huggingface_hub>=0.25.0
+Provides-Extra: quality
+Requires-Dist: black; extra == "quality"
+Requires-Dist: hf-doc-builder; extra == "quality"
+Requires-Dist: ruff~=0.12.8; extra == "quality"
+Provides-Extra: docs-specific
+Requires-Dist: black; extra == "docs-specific"
+Requires-Dist: hf-doc-builder; extra == "docs-specific"
+Provides-Extra: dev
+Requires-Dist: black; extra == "dev"
+Requires-Dist: hf-doc-builder; extra == "dev"
+Requires-Dist: ruff~=0.12.8; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: hf-doc-builder; extra == "dev"
+Provides-Extra: test
+Requires-Dist: black; extra == "test"
+Requires-Dist: hf-doc-builder; extra == "test"
+Requires-Dist: ruff~=0.12.8; extra == "test"
+Requires-Dist: black; extra == "test"
+Requires-Dist: hf-doc-builder; extra == "test"
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+Requires-Dist: pytest-xdist; extra == "test"
+Requires-Dist: parameterized; extra == "test"
+Requires-Dist: datasets; extra == "test"
+Requires-Dist: diffusers; extra == "test"
+Requires-Dist: scipy; extra == "test"
+Requires-Dist: protobuf; extra == "test"
+Requires-Dist: sentencepiece; extra == "test"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+<!---
+Copyright 2023 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+<h1 align="center"> <p>🤗 PEFT</p></h1>
+<h3 align="center">
+    <p>State-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methods</p>
+</h3>
+
+Fine-tuning large pretrained models is often prohibitively costly due to their scale. Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of large pretrained models to various downstream applications by only fine-tuning a small number of (extra) model parameters instead of all the model's parameters. This significantly decreases the computational and storage costs. Recent state-of-the-art PEFT techniques achieve performance comparable to fully fine-tuned models.
+
+PEFT is integrated with Transformers for easy model training and inference, Diffusers for conveniently managing different adapters, and Accelerate for distributed training and inference for really big models.
+
+> [!TIP]
+> Visit the [PEFT](https://huggingface.co/PEFT) organization to read about the PEFT methods implemented in the library and to see notebooks demonstrating how to apply these methods to a variety of downstream tasks. Click the "Watch repos" button on the organization page to be notified of newly implemented methods and notebooks!
+
+Check the PEFT Adapters API Reference section for a list of supported PEFT methods, and read the [Adapters](https://huggingface.co/docs/peft/en/conceptual_guides/adapter), [Soft prompts](https://huggingface.co/docs/peft/en/conceptual_guides/prompting), and [IA3](https://huggingface.co/docs/peft/en/conceptual_guides/ia3) conceptual guides to learn more about how these methods work.
+
+## Quickstart
+
+Install PEFT from pip:
+
+```bash
+pip install peft
+```
+
+Prepare a model for training with a PEFT method such as LoRA by wrapping the base model and PEFT configuration with `get_peft_model`. For the bigscience/mt0-large model, you're only training 0.19% of the parameters!
+
+```python
+from transformers import AutoModelForCausalLM
+from peft import LoraConfig, TaskType, get_peft_model
+
+device = torch.accelerator.current_accelerator().type if hasattr(torch, "accelerator") else "cuda"
+model_id = "Qwen/Qwen2.5-3B-Instruct"
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map=device)
+peft_config = LoraConfig(
+    r=16,
+    lora_alpha=32,
+    task_type=TaskType.CAUSAL_LM,
+    # target_modules=["q_proj", "v_proj", ...]  # optionally indicate target modules
+)
+model = get_peft_model(model, peft_config)
+model.print_trainable_parameters()
+# prints: trainable params: 3,686,400 || all params: 3,089,625,088 || trainable%: 0.1193
+
+# now perform training on your dataset, e.g. using transformers Trainer, then save the model
+model.save_pretrained("qwen2.5-3b-lora")
+```
+
+To load a PEFT model for inference:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+device = torch.accelerator.current_accelerator().type if hasattr(torch, "accelerator") else "cuda"
+model_id = "Qwen/Qwen2.5-3B-Instruct"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map=device)
+model = PeftModel.from_pretrained(model, "qwen2.5-3b-lora")
+
+inputs = tokenizer("Preheat the oven to 350 degrees and place the cookie dough", return_tensors="pt")
+outputs = model.generate(**inputs.to(device), max_new_tokens=50)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+
+# prints something like: Preheat the oven to 350 degrees and place the cookie dough in a baking dish [...]
+```
+
+## Why you should use PEFT
+
+There are many benefits of using PEFT but the main one is the huge savings in compute and storage, making PEFT applicable to many different use cases.
+
+### High performance on consumer hardware
+
+Consider the memory requirements for training the following models on the [ought/raft/twitter_complaints](https://huggingface.co/datasets/ought/raft/viewer/twitter_complaints) dataset with an A100 80GB GPU with more than 64GB of CPU RAM.
+
+|   Model         | Full Finetuning | PEFT-LoRA PyTorch  | PEFT-LoRA DeepSpeed with CPU Offloading |
+| --------- | ---- | ---- | ---- |
+| bigscience/T0_3B (3B params) | 47.14GB GPU / 2.96GB CPU  | 14.4GB GPU / 2.96GB CPU | 9.8GB GPU / 17.8GB CPU |
+| bigscience/mt0-xxl (12B params) | OOM GPU | 56GB GPU / 3GB CPU | 22GB GPU / 52GB CPU |
+| bigscience/bloomz-7b1 (7B params) | OOM GPU | 32GB GPU / 3.8GB CPU | 18.1GB GPU / 35GB CPU |
+
+With LoRA you can fully finetune a 12B parameter model that would've otherwise run out of memory on the 80GB GPU, and comfortably fit and train a 3B parameter model. When you look at the 3B parameter model's performance, it is comparable to a fully finetuned model at a fraction of the GPU memory.
+
+|   Submission Name        | Accuracy |
+| --------- | ---- |
+| Human baseline (crowdsourced) |	0.897 |
+| Flan-T5 | 0.892 |
+| lora-t0-3b | 0.863 |
+
+> [!TIP]
+> The bigscience/T0_3B model performance isn't optimized in the table above. You can squeeze even more performance out of it by playing around with the input instruction templates, LoRA hyperparameters, and other training related hyperparameters. The final checkpoint size of this model is just 19MB compared to 11GB of the full bigscience/T0_3B model. Learn more about the advantages of finetuning with PEFT in this [blog post](https://www.philschmid.de/fine-tune-flan-t5-peft).
+
+### Quantization
+
+Quantization is another method for reducing the memory requirements of a model by representing the data in a lower precision. It can be combined with PEFT methods to make it even easier to train and load LLMs for inference.
+
+* Learn how to finetune [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) with QLoRA and the [TRL](https://huggingface.co/docs/trl/index) library on a 16GB GPU in the [Finetune LLMs on your own consumer hardware using tools from PyTorch and Hugging Face ecosystem](https://pytorch.org/blog/finetune-llms/) blog post.
+* Learn how to finetune a [openai/whisper-large-v2](https://huggingface.co/openai/whisper-large-v2) model for multilingual automatic speech recognition with LoRA and 8-bit quantization in this [notebook](https://colab.research.google.com/drive/1DOkD_5OUjFa0r5Ik3SgywJLJtEo2qLxO?usp=sharing) (see this [notebook](https://colab.research.google.com/drive/1vhF8yueFqha3Y3CpTHN6q9EVcII9EYzs?usp=sharing) instead for an example of streaming a dataset).
+
+### Save compute and storage
+
+PEFT can help you save storage by avoiding full finetuning of models on each of downstream task or dataset. In many cases, you're only finetuning a very small fraction of a model's parameters and each checkpoint is only a few MBs in size (instead of GBs). These smaller PEFT adapters demonstrate performance comparable to a fully finetuned model. If you have many datasets, you can save a lot of storage with a PEFT model and not have to worry about catastrophic forgetting or overfitting the backbone or base model.
+
+## PEFT integrations
+
+PEFT is widely supported across the Hugging Face ecosystem because of the massive efficiency it brings to training and inference.
+
+### Diffusers
+
+The iterative diffusion process consumes a lot of memory which can make it difficult to train. PEFT can help reduce the memory requirements and reduce the storage size of the final model checkpoint. For example, consider the memory required for training a Stable Diffusion model with LoRA on an A100 80GB GPU with more than 64GB of CPU RAM. The final model checkpoint size is only 8.8MB!
+
+|   Model         | Full Finetuning | PEFT-LoRA  | PEFT-LoRA with Gradient Checkpointing  |
+| --------- | ---- | ---- | ---- |
+| CompVis/stable-diffusion-v1-4 | 27.5GB GPU / 3.97GB CPU | 15.5GB GPU / 3.84GB CPU | 8.12GB GPU / 3.77GB CPU | 
+
+> [!TIP]
+> Take a look at the [examples/lora_dreambooth/train_dreambooth.py](examples/lora_dreambooth/train_dreambooth.py) training script to try training your own Stable Diffusion model with LoRA, and play around with the [smangrul/peft-lora-sd-dreambooth](https://huggingface.co/spaces/smangrul/peft-lora-sd-dreambooth) Space which is running on a T4 instance. Learn more about the PEFT integration in Diffusers in this [tutorial](https://huggingface.co/docs/peft/main/en/tutorial/peft_integrations#diffusers).
+
+### Transformers
+
+PEFT is directly integrated with [Transformers](https://huggingface.co/docs/transformers/main/en/peft). After loading a model, call `add_adapter` to add a new PEFT adapter to the model:
+
+```python
+from peft import LoraConfig
+model = ...  # transformers model
+peft_config = LoraConfig(...)
+model.add_adapter(lora_config, adapter_name="lora_1")
+```
+
+To load a trained PEFT adapter, call `load_adapter`:
+
+```python
+model = ...  # transformers model
+model.load_adapter(<path-to-adapter>, adapter_name="lora_1")
+```
+
+And to switch between different adapters, call `set_adapter`:
+
+```python
+model.set_adapter("lora_2")
+```
+
+The Transformers integration doesn't include all the functionalities offered in PEFT, such as methods for merging the adapter into the base model.
+
+### Accelerate
+
+[Accelerate](https://huggingface.co/docs/accelerate/index) is a library for distributed training and inference on various training setups and hardware (GPUs, TPUs, Apple Silicon, etc.). PEFT models work with Accelerate out of the box, making it really convenient to train really large models or use them for inference on consumer hardware with limited resources.
+
+### TRL
+
+PEFT can also be applied to training LLMs with RLHF components such as the ranker and policy. Get started by reading:
+
+* [Fine-tune a Mistral-7b model with Direct Preference Optimization](https://towardsdatascience.com/fine-tune-a-mistral-7b-model-with-direct-preference-optimization-708042745aac) with PEFT and the [TRL](https://huggingface.co/docs/trl/index) library to learn more about the Direct Preference Optimization (DPO) method and how to apply it to a LLM.
+* [Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU](https://huggingface.co/blog/trl-peft) with PEFT and the [TRL](https://huggingface.co/docs/trl/index) library, and then try out the [gpt2-sentiment_peft.ipynb](https://github.com/huggingface/trl/blob/main/examples/notebooks/gpt2-sentiment.ipynb) notebook to optimize GPT2 to generate positive movie reviews.
+* [StackLLaMA: A hands-on guide to train LLaMA with RLHF](https://huggingface.co/blog/stackllama) with PEFT, and then try out the [stack_llama/scripts](https://github.com/huggingface/trl/tree/main/examples/research_projects/stack_llama/scripts) for supervised finetuning, reward modeling, and RL finetuning.
+
+## Model support
+
+Use this [Space](https://stevhliu-peft-methods.hf.space) or check out the [docs](https://huggingface.co/docs/peft/main/en/index) to find which models officially support a PEFT method out of the box. Even if you don't see a model listed below, you can manually configure the model config to enable PEFT for a model. Read the [New transformers architecture](https://huggingface.co/docs/peft/main/en/developer_guides/custom_models#new-transformers-architectures) guide to learn how.
+
+## Contribute
+
+If you would like to contribute to PEFT, please check out our [contribution guide](https://huggingface.co/docs/peft/developer_guides/contributing).
+
+## Citing 🤗 PEFT
+
+To use 🤗 PEFT in your publication, please cite it by using the following BibTeX entry.
+
+```bibtex
+@Misc{peft,
+  title =        {{PEFT}: State-of-the-art Parameter-Efficient Fine-Tuning methods},
+  author =       {Sourab Mangrulkar and Sylvain Gugger and Lysandre Debut and Younes Belkada and Sayak Paul and Benjamin Bossan},
+  howpublished = {\url{https://github.com/huggingface/peft}},
+  year =         {2022}
+}
+```