Upload fine-tuned model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,845 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:128997
+- loss:MultipleNegativesRankingLoss
+base_model: suhwan3/mpnet_step1
+widget:
+- source_sentence: The Global X S&P 500 Risk Managed Income ETF seeks to track the
+    Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets
+    in index securities. The index's strategy involves holding the underlying stocks
+    of the S&P 500 Index while applying an options collar, specifically selling at-the-money
+    covered call options and buying monthly 5% out-of-the-money put options corresponding
+    to the portfolio's value. This approach aims to generate income, ideally resulting
+    in a net credit from the options premiums, and provide risk management, though
+    selling at-the-money calls inherently caps the fund's potential for upside participation.
+  sentences:
+  - Nasdaq, Inc. operates as a technology company that serves capital markets and
+    other industries worldwide. The Market Technology segment includes anti financial
+    crime technology business, which offers Nasdaq Trade Surveillance, a SaaS solution
+    for brokers and other market participants to assist them in complying with market
+    rules, regulations, and internal market surveillance policies; Nasdaq Automated
+    Investigator, a cloud-deployed anti-money laundering tool; and Verafin, a SaaS
+    technology provider of anti-financial crime management solutions. This segment
+    also handles assets, such as cash equities, equity derivatives, currencies, interest-bearing
+    securities, commodities, energy products, and digital currencies. The Investment
+    Intelligence segment sells and distributes historical and real-time market data;
+    develops and licenses Nasdaq-branded indexes and financial products; and provides
+    investment insights and workflow solutions. The Corporate Platforms segment operates
+    listing platforms; and offers investor relations intelligence and governance solutions.
+    As of December 31, 2021, it had 4,178 companies listed securities on The Nasdaq
+    Stock Market, including 1,632 listings on The Nasdaq Global Select Market; 1,169
+    on The Nasdaq Global Market; and 1,377 on The Nasdaq Capital Market. The Market
+    Services segment includes equity derivative trading and clearing, cash equity
+    trading, fixed income and commodities trading and clearing, and trade management
+    service businesses. This segment operates various exchanges and other marketplace
+    facilities across various asset classes, which include derivatives, commodities,
+    cash equity, debt, structured products, and exchange traded products; and provides
+    broker, clearing, settlement, and central depository services. The company was
+    formerly known as The NASDAQ OMX Group, Inc. and changed its name to Nasdaq, Inc.
+    in September 2015. Nasdaq, Inc. was founded in 1971 and is headquartered in New
+    York, New York.
+  - Jabil Inc. provides manufacturing services and solutions worldwide. The company
+    operates in two segments, Electronics Manufacturing Services and Diversified Manufacturing
+    Services. It offers electronics design, production, and product management services.
+    The company provides electronic design services, such as application-specific
+    integrated circuit design, firmware development, and rapid prototyping services;
+    and designs plastic and metal enclosures that include the electro-mechanics, such
+    as the printed circuit board assemblies (PCBA). It also specializes in the three-dimensional
+    mechanical design comprising the analysis of electronic, electro-mechanical, and
+    optical assemblies, as well as offers various industrial design, mechanism development,
+    and tooling management services. In addition, the company provides computer-assisted
+    design services consisting of PCBA design, as well as PCBA design validation and
+    verification services; and other consulting services, such as the generation of
+    a bill of materials, approved vendor list, and assembly equipment configuration
+    for various PCBA designs. Further, it offers product and process validation services,
+    such as product system, product safety, regulatory compliance, and reliability
+    tests, as well as manufacturing test solution development services. Additionally,
+    the company provides systems assembly, test, direct-order fulfillment, and configure-to-order
+    services. It serves 5G, wireless and cloud, digital print and retail, industrial
+    and semi-cap, networking and storage, automotive and transportation, connected
+    devices, healthcare and packaging, and mobility industries. The company was formerly
+    known as Jabil Circuit, Inc. and changed its name to Jabil Inc. in June 2017.
+    Jabil Inc. was founded in 1966 and is headquartered in Saint Petersburg, Florida.
+  - 'Realty Income, The Monthly Dividend Company, is an S&P 500 company dedicated
+    to providing stockholders with dependable monthly income. The company is structured
+    as a REIT, and its monthly dividends are supported by the cash flow from over
+    6,500 real estate properties owned under long-term lease agreements with our commercial
+    clients. To date, the company has declared 608 consecutive common stock monthly
+    dividends throughout its 52-year operating history and increased the dividend
+    109 times since Realty Income''s public listing in 1994 (NYSE: O). The company
+    is a member of the S&P 500 Dividend Aristocrats index. Additional information
+    about the company can be obtained from the corporate website at www.realtyincome.com.'
+- source_sentence: The iShares U.S. Telecommunications ETF (IYZ) seeks to track the
+    investment results of the Russell 1000 Telecommunications RIC 22.5/45 Capped Index,
+    which measures the performance of the U.S. telecommunications sector of the U.S.
+    equity market as defined by FTSE Russell. This market-cap-weighted index includes
+    large-cap companies involved in telecom equipment and service provision and is
+    subject to regulatory capping that limits single holdings to 22.5% and aggregate
+    large holdings to 45%. The fund generally invests at least 80% of its assets in
+    the component securities of its underlying index and is non-diversified; the underlying
+    index is rebalanced quarterly.
+  sentences:
+  - Kanzhun Limited operates an online recruitment platform, BOSS Zhipin in the People's
+    Republic of China. Its recruitment platform assists the recruitment process between
+    job seekers and employers for enterprises, and corporations. The company was founded
+    in 2013 and is headquartered in Beijing, the People's Republic of China.
+  - Frontier Communications Parent, Inc., together with its subsidiaries, provides
+    communications services for consumer and business customers in 25 states in the
+    United States. It offers data and Internet, voice, video, and other services.
+    The company was formerly known as Frontier Communications Corporation and changed
+    its name to Frontier Communications Parent, Inc. in April 2021. Frontier Communications
+    Parent, Inc. was incorporated in 1935 and is based in Norwalk, Connecticut.
+  - Broadcom Inc. designs, develops, and supplies various semiconductor devices with
+    a focus on complex digital and mixed signal complementary metal oxide semiconductor
+    based devices and analog III-V based products worldwide. The company operates
+    in two segments, Semiconductor Solutions and Infrastructure Software. It provides
+    set-top box system-on-chips (SoCs); cable, digital subscriber line, and passive
+    optical networking central office/consumer premise equipment SoCs; wireless local
+    area network access point SoCs; Ethernet switching and routing merchant silicon
+    products; embedded processors and controllers; serializer/deserializer application
+    specific integrated circuits; optical and copper, and physical layers; and fiber
+    optic transmitter and receiver components. The company also offers RF front end
+    modules, filters, and power amplifiers; Wi-Fi, Bluetooth, and global positioning
+    system/global navigation satellite system SoCs; custom touch controllers; serial
+    attached small computer system interface, and redundant array of independent disks
+    controllers and adapters; peripheral component interconnect express switches;
+    fiber channel host bus adapters; read channel based SoCs; custom flash controllers;
+    preamplifiers; and optocouplers, industrial fiber optics, and motion control encoders
+    and subsystems. Its products are used in various applications, including enterprise
+    and data center networking, home connectivity, set-top boxes, broadband access,
+    telecommunication equipment, smartphones and base stations, data center servers
+    and storage systems, factory automation, power generation and alternative energy
+    systems, and electronic displays. Broadcom Inc. was incorporated in 2018 and is
+    headquartered in San Jose, California.
+- source_sentence: The Xtrackers MSCI Emerging Markets ESG Leaders Equity ETF tracks
+    an index of large- and mid-cap emerging market stocks that emphasize strong environmental,
+    social, and governance (ESG) characteristics. The index first excludes companies
+    involved in specific controversial industries. From the remaining universe, it
+    ranks stocks based on MSCI ESG scores, including a controversy component, to identify
+    and select the highest-ranking ESG leaders, effectively screening out ESG laggards.
+    To maintain market-like country and sector weights, the index selects the top
+    ESG-scoring stocks within each sector until a specified market capitalization
+    threshold is reached. Selected stocks are then weighted by market capitalization
+    within their respective sectors. The fund typically invests over 80% of its assets
+    in the securities of this underlying index.
+  sentences:
+  - Info Edge (India) Limited operates as an online classifieds company in the areas
+    of recruitment, matrimony, real estate, and education and related services in
+    India and internationally. It operates through Recruitment Solutions, 99acres,
+    and Other segments. The company offers recruitment services through naukri.com,
+    an online job website for job seekers and corporate customers, including hiring
+    consultants; firstnaukri.com, a job search network for college students and recent
+    graduates; naukrigulf.com, a website catering to Gulf markets; and quadranglesearch.com,
+    a site that provides off-line placement services to middle and senior management,
+    as well as Highorbit/iimjobs.com, zwayam.com, hirist.com, doselect.com, ambitionbox.com,
+    bigshyft.com, and jobhai.com. It also provides 99acres.com, which offers listing
+    of properties for sale, purchase, and rent; Jeevansathi.com, an online matrimonial
+    classifieds services; and shiksha.com, an education classified website that helps
+    students to decide their undergraduate and postgraduate options by providing useful
+    information on careers, exams, colleges, and courses, as well as operates multiple
+    dating platforms on the web through its mobile apps Aisle, Anbe, Arike and HeyDil.
+    In addition, the company provides internet, computer, and electronic and related
+    services; and software development, consultancy, technical support for consumer
+    companies, SAAS providers, and other services in the field of information technology
+    and product development, as well as brokerage services in the real estate sector.
+    Further, it acts as an investment adviser and manager, financial and management
+    consultant, and sponsor of alternative investment funds, as well as provides advertising
+    space for colleges and universities on www.shiksha.com. Info Edge (India) Limited
+    was incorporated in 1995 and is based in Noida, India.
+  - China Overseas Land & Investment Limited, an investment holding company, engages
+    in the property development and investment, and other operations in the People's
+    Republic of China and the United Kingdom. The company operates through Property
+    Development, Property Investment, and Other Operations segments. It is involved
+    in the investment, development, and rental of residential and commercial properties;
+    issuance of guaranteed notes and corporate bonds; and hotel operation activities.
+    The company also provides construction and building design consultancy services.
+    In addition, it engages in the investment and financing, land consolidation, regional
+    planning, engineering construction, industrial import, commercial operation, and
+    property management. Further, the company offers urban services, including office
+    buildings, flexible working space, shopping malls, star-rated hotels, long-term
+    rental apartments, logistics parks, and architectural design and construction.
+    The company was founded in 1979 and is based in Central, Hong Kong. China Overseas
+    Land & Investment Limited is a subsidiary of China Overseas Holdings Limited.
+  - Mastercard Incorporated, a technology company, provides transaction processing
+    and other payment-related products and services in the United States and internationally.
+    It facilitates the processing of payment transactions, including authorization,
+    clearing, and settlement, as well as delivers other payment-related products and
+    services. The company offers integrated products and value-added services for
+    account holders, merchants, financial institutions, businesses, governments, and
+    other organizations, such as programs that enable issuers to provide consumers
+    with credits to defer payments; prepaid programs and management services; commercial
+    credit and debit payment products and solutions; and payment products and solutions
+    that allow its customers to access funds in deposit and other accounts. It also
+    provides value-added products and services comprising cyber and intelligence solutions
+    for parties to transact, as well as proprietary insights, drawing on principled
+    use of consumer, and merchant data services. In addition, the company offers analytics,
+    test and learn, consulting, managed services, loyalty, processing, and payment
+    gateway solutions for e-commerce merchants. Further, it provides open banking
+    and digital identity platforms services. The company offers payment solutions
+    and services under the MasterCard, Maestro, and Cirrus. Mastercard Incorporated
+    was founded in 1966 and is headquartered in Purchase, New York.
+- source_sentence: The Global X S&P 500 Risk Managed Income ETF seeks to track the
+    Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets
+    in index securities. The index's strategy involves holding the underlying stocks
+    of the S&P 500 Index while applying an options collar, specifically selling at-the-money
+    covered call options and buying monthly 5% out-of-the-money put options corresponding
+    to the portfolio's value. This approach aims to generate income, ideally resulting
+    in a net credit from the options premiums, and provide risk management, though
+    selling at-the-money calls inherently caps the fund's potential for upside participation.
+  sentences:
+  - Incyte Corporation, a biopharmaceutical company, focuses on the discovery, development,
+    and commercialization of proprietary therapeutics in the United States and internationally.
+    The company offers JAKAFI, a drug for the treatment of myelofibrosis and polycythemia
+    vera; PEMAZYRE, a fibroblast growth factor receptor kinase inhibitor that act
+    as oncogenic drivers in various liquid and solid tumor types; and ICLUSIG, a kinase
+    inhibitor to treat chronic myeloid leukemia and philadelphia-chromosome positive
+    acute lymphoblastic leukemia. Its clinical stage products include ruxolitinib,
+    a steroid-refractory chronic graft-versus-host-diseases (GVHD); itacitinib, which
+    is in Phase II/III clinical trial to treat naive chronic GVHD; and pemigatinib
+    for treating bladder cancer, cholangiocarcinoma, myeloproliferative syndrome,
+    and tumor agnostic. In addition, the company engages in developing Parsaclisib,
+    which is in Phase II clinical trial for follicular lymphoma, marginal zone lymphoma,
+    and mantel cell lymphoma. Additionally, it develops Retifanlimab that is in Phase
+    II clinical trials for MSI-high endometrial cancer, merkel cell carcinoma, and
+    anal cancer, as well as in Phase II clinical trials for patients with non-small
+    cell lung cancer. It has collaboration agreements with Novartis International
+    Pharmaceutical Ltd.; Eli Lilly and Company; Agenus Inc.; Calithera Biosciences,
+    Inc; MacroGenics, Inc.; Merus N.V.; Syros Pharmaceuticals, Inc.; Innovent Biologics,
+    Inc.; Zai Lab Limited; Cellenkos, Inc.; and Nimble Therapeutics, as well as clinical
+    collaborations with MorphoSys AG and Xencor, Inc. to investigate the combination
+    of tafasitamab, plamotamab, and lenalidomide in patients with relapsed or refractory
+    diffuse large B-cell lymphoma, and relapsed or refractory follicular lymphoma.
+    The company was incorporated in 1991 and is headquartered in Wilmington, Delaware.
+  - Omnicom Group Inc., together with its subsidiaries, provides advertising, marketing,
+    and corporate communications services. It provides a range of services in the
+    areas of advertising, customer relationship management, public relations, and
+    healthcare. The company's services include advertising, branding, content marketing,
+    corporate social responsibility consulting, crisis communications, custom publishing,
+    data analytics, database management, digital/direct marketing, digital transformation,
+    entertainment marketing, experiential marketing, field marketing, financial/corporate
+    business-to-business advertising, graphic arts/digital imaging, healthcare marketing
+    and communications, and in-store design services. Its services also comprise interactive
+    marketing, investor relations, marketing research, media planning and buying,
+    merchandising and point of sale, mobile marketing, multi-cultural marketing, non-profit
+    marketing, organizational communications, package design, product placement, promotional
+    marketing, public affairs, retail marketing, sales support, search engine marketing,
+    shopper marketing, social media marketing, and sports and event marketing services.
+    It operates in the United States, Canada, Puerto Rico, South America, Mexico,
+    Europe, the Middle East, Africa, Australia, Greater China, India, Japan, Korea,
+    New Zealand, Singapore, and other Asian countries. The company was incorporated
+    in 1944 and is based in New York, New York.
+  - NetApp, Inc. provides cloud-led and data-centric services to manage and share
+    data on-premises, and private and public clouds worldwide. It operates in two
+    segments, Hybrid Cloud and Public Could. The company offers intelligent data management
+    software, such as NetApp ONTAP, NetApp Snapshot, NetApp SnapCenter Backup Management,
+    NetApp SnapMirror Data Replication, NetApp SnapLock Data Compliance, NetApp ElementOS
+    software, and NetApp SANtricity software; and storage infrastructure solutions,
+    including NetApp All-Flash FAS series, NetApp Fabric Attached Storage, NetApp
+    FlexPod, NetApp E/EF series, NetApp StorageGRID, and NetApp SolidFire. It also
+    provides cloud storage and data services comprising NetApp Cloud Volumes ONTAP,
+    Azure NetApp Files, Amazon FSx for NetApp ONTAP, NetApp Cloud Volumes Service
+    for Google Cloud, NetApp Cloud Sync, NetApp Cloud Tiering, NetApp Cloud Backup,
+    NetApp Cloud Data Sense, and NetApp Cloud Volumes Edge Cache; and cloud operations
+    services, such as NetApp Cloud Insights, Spot Ocean Kubernetes Suite, Spot Security,
+    Spot Eco, and Spot CloudCheckr. In addition, the company offers application-aware
+    data management service under the NetApp Astra name; and professional and support
+    services, such as strategic consulting, professional, managed, and support services.
+    Further, it provides assessment, design, implementation, and migration services.
+    The company serves the energy, financial service, government, technology, internet,
+    life science, healthcare service, manufacturing, media, entertainment, animation,
+    video postproduction, and telecommunication markets through a direct sales force
+    and an ecosystem of partners. NetApp, Inc. was incorporated in 1992 and is headquartered
+    in San Jose, California.
+- source_sentence: The Global X S&P 500 Risk Managed Income ETF seeks to track the
+    Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets
+    in index securities. The index's strategy involves holding the underlying stocks
+    of the S&P 500 Index while applying an options collar, specifically selling at-the-money
+    covered call options and buying monthly 5% out-of-the-money put options corresponding
+    to the portfolio's value. This approach aims to generate income, ideally resulting
+    in a net credit from the options premiums, and provide risk management, though
+    selling at-the-money calls inherently caps the fund's potential for upside participation.
+  sentences:
+  - Walgreens Boots Alliance, Inc. operates as a pharmacy-led health and beauty retail
+    company. It operates through two segments, the United States and International.
+    The United States segment sells prescription drugs and an assortment of retail
+    products, including health, wellness, beauty, personal care, consumable, and general
+    merchandise products through its retail drugstores. It also provides central specialty
+    pharmacy services and mail services. As of August 31, 2021, this segment operated
+    8,965 retail stores under the Walgreens and Duane Reade brands in the United States;
+    and five specialty pharmacies. The International segment sells prescription drugs;
+    and health and wellness, beauty, personal care, and other consumer products through
+    its pharmacy-led health and beauty retail stores and optical practices, as well
+    as through boots.com and an integrated mobile application. It also engages in
+    pharmaceutical wholesaling and distribution business in Germany. As of August
+    31, 2021, this segment operated 4,031 retail stores under the Boots, Benavides,
+    and Ahumada in the United Kingdom, Thailand, Norway, the Republic of Ireland,
+    the Netherlands, Mexico, and Chile; and 548 optical practices, including 160 on
+    a franchise basis. Walgreens Boots Alliance, Inc. was founded in 1901 and is based
+    in Deerfield, Illinois.
+  - Middlesex Water Company owns and operates regulated water utility and wastewater
+    systems. It operates in two segments, Regulated and Non-Regulated. The Regulated
+    segment collects, treats, and distributes water on a retail and wholesale basis
+    to residential, commercial, industrial, and fire protection customers, as well
+    as provides regulated wastewater systems in New Jersey and Delaware. The Non-Regulated
+    segment provides non-regulated contract services for the operation and maintenance
+    of municipal and private water and wastewater systems in New Jersey and Delaware.
+    The company was incorporated in 1896 and is headquartered in Iselin, New Jersey.
+  - Liberty Broadband Corporation engages in the communications businesses. It operates
+    through GCI Holdings and Charter segments. The GCI Holdings segment provides a
+    range of wireless, data, video, voice, and managed services to residential customers,
+    businesses, governmental entities, and educational and medical institutions primarily
+    in Alaska under the GCI brand. The Charter segment offers subscription-based video
+    services comprising video on demand, high-definition television, and digital video
+    recorder service; local and long-distance calling, voicemail, call waiting, caller
+    ID, call forwarding, and other voice services, as well as international calling
+    services; and Spectrum TV. It also provides internet services, including an in-home
+    Wi-Fi product that provides customers with high-performance wireless routers and
+    managed Wi-Fi services; advanced community Wi-Fi; mobile internet; and a security
+    suite that offers protection against computer viruses and spyware. In addition,
+    this segment offers internet access, data networking, fiber connectivity to cellular
+    towers and office buildings, video entertainment, and business telephone services;
+    advertising services on cable television networks and digital outlets; and operates
+    regional sports and news networks. Liberty Broadband Corporation was incorporated
+    in 2014 and is based in Englewood, Colorado.
+datasets:
+- hobbang/stage2-dataset
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+---
+
+# SentenceTransformer based on suhwan3/mpnet_step1
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [suhwan3/mpnet_step1](https://huggingface.co/suhwan3/mpnet_step1) on the [stage2-dataset](https://huggingface.co/datasets/hobbang/stage2-dataset) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [suhwan3/mpnet_step1](https://huggingface.co/suhwan3/mpnet_step1) <!-- at revision 8857c26669998d56b0735085b269cfc7890ca67d -->
+- **Maximum Sequence Length:** 384 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [stage2-dataset](https://huggingface.co/datasets/hobbang/stage2-dataset)
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    "The Global X S&P 500 Risk Managed Income ETF seeks to track the Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets in index securities. The index's strategy involves holding the underlying stocks of the S&P 500 Index while applying an options collar, specifically selling at-the-money covered call options and buying monthly 5% out-of-the-money put options corresponding to the portfolio's value. This approach aims to generate income, ideally resulting in a net credit from the options premiums, and provide risk management, though selling at-the-money calls inherently caps the fund's potential for upside participation.",
+    'Walgreens Boots Alliance, Inc. operates as a pharmacy-led health and beauty retail company. It operates through two segments, the United States and International. The United States segment sells prescription drugs and an assortment of retail products, including health, wellness, beauty, personal care, consumable, and general merchandise products through its retail drugstores. It also provides central specialty pharmacy services and mail services. As of August 31, 2021, this segment operated 8,965 retail stores under the Walgreens and Duane Reade brands in the United States; and five specialty pharmacies. The International segment sells prescription drugs; and health and wellness, beauty, personal care, and other consumer products through its pharmacy-led health and beauty retail stores and optical practices, as well as through boots.com and an integrated mobile application. It also engages in pharmaceutical wholesaling and distribution business in Germany. As of August 31, 2021, this segment operated 4,031 retail stores under the Boots, Benavides, and Ahumada in the United Kingdom, Thailand, Norway, the Republic of Ireland, the Netherlands, Mexico, and Chile; and 548 optical practices, including 160 on a franchise basis. Walgreens Boots Alliance, Inc. was founded in 1901 and is based in Deerfield, Illinois.',
+    'Liberty Broadband Corporation engages in the communications businesses. It operates through GCI Holdings and Charter segments. The GCI Holdings segment provides a range of wireless, data, video, voice, and managed services to residential customers, businesses, governmental entities, and educational and medical institutions primarily in Alaska under the GCI brand. The Charter segment offers subscription-based video services comprising video on demand, high-definition television, and digital video recorder service; local and long-distance calling, voicemail, call waiting, caller ID, call forwarding, and other voice services, as well as international calling services; and Spectrum TV. It also provides internet services, including an in-home Wi-Fi product that provides customers with high-performance wireless routers and managed Wi-Fi services; advanced community Wi-Fi; mobile internet; and a security suite that offers protection against computer viruses and spyware. In addition, this segment offers internet access, data networking, fiber connectivity to cellular towers and office buildings, video entertainment, and business telephone services; advertising services on cable television networks and digital outlets; and operates regional sports and news networks. Liberty Broadband Corporation was incorporated in 2014 and is based in Englewood, Colorado.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### stage2-dataset
+
+* Dataset: [stage2-dataset](https://huggingface.co/datasets/hobbang/stage2-dataset) at [cd393c2](https://huggingface.co/datasets/hobbang/stage2-dataset/tree/cd393c24f4017971e95aa6f73736f2fcb45e30a0)
+* Size: 128,997 training samples
+* Columns: <code>anchor</code> and <code>positive</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                                | positive                                                                             |
+  |:--------|:--------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                                | string                                                                               |
+  | details | <ul><li>min: 101 tokens</li><li>mean: 143.15 tokens</li><li>max: 186 tokens</li></ul> | <ul><li>min: 35 tokens</li><li>mean: 238.69 tokens</li><li>max: 384 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+  |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>The Invesco Financial Preferred ETF (PGF) seeks to track the ICE Exchange-Listed Fixed Rate Financial Preferred Securities Index, primarily by investing at least 90% of its total assets in the securities comprising the index. The underlying index is market capitalization weighted and designed to track the performance of exchange-listed, fixed rate, U.S. dollar denominated preferred securities, including functionally equivalent instruments, issued by U.S. financial companies. PGF provides a concentrated portfolio exclusively focused on financial-sector preferred securities and is considered non-diversified, holding both investment- and non-investment-grade securities within this focus.</code> | <code>JPMorgan Chase & Co. operates as a financial services company worldwide. It operates through four segments: Consumer & Community Banking (CCB), Corporate & Investment Bank (CIB), Commercial Banking (CB), and Asset & Wealth Management (AWM). The CCB segment offers s deposit, investment and lending products, payments, and services to consumers; lending, deposit, and cash management and payment solutions to small businesses; mortgage origination and servicing activities; residential mortgages and home equity loans; and credit card, auto loan, and leasing services. The CIB segment provides investment banking products and services, including corporate strategy and structure advisory, and equity and debt markets capital-raising services, as well as loan origination and syndication; payments and cross-border financing; and cash and derivative instruments, risk management solutions, prime brokerage, and research. This segment also offers securities services, including custody, fund accounting ...</code> |
+  | <code>The Invesco Financial Preferred ETF (PGF) seeks to track the ICE Exchange-Listed Fixed Rate Financial Preferred Securities Index, primarily by investing at least 90% of its total assets in the securities comprising the index. The underlying index is market capitalization weighted and designed to track the performance of exchange-listed, fixed rate, U.S. dollar denominated preferred securities, including functionally equivalent instruments, issued by U.S. financial companies. PGF provides a concentrated portfolio exclusively focused on financial-sector preferred securities and is considered non-diversified, holding both investment- and non-investment-grade securities within this focus.</code> | <code>JPMorgan Chase & Co. operates as a financial services company worldwide. It operates through four segments: Consumer & Community Banking (CCB), Corporate & Investment Bank (CIB), Commercial Banking (CB), and Asset & Wealth Management (AWM). The CCB segment offers s deposit, investment and lending products, payments, and services to consumers; lending, deposit, and cash management and payment solutions to small businesses; mortgage origination and servicing activities; residential mortgages and home equity loans; and credit card, auto loan, and leasing services. The CIB segment provides investment banking products and services, including corporate strategy and structure advisory, and equity and debt markets capital-raising services, as well as loan origination and syndication; payments and cross-border financing; and cash and derivative instruments, risk management solutions, prime brokerage, and research. This segment also offers securities services, including custody, fund accounting ...</code> |
+  | <code>The Invesco Financial Preferred ETF (PGF) seeks to track the ICE Exchange-Listed Fixed Rate Financial Preferred Securities Index, primarily by investing at least 90% of its total assets in the securities comprising the index. The underlying index is market capitalization weighted and designed to track the performance of exchange-listed, fixed rate, U.S. dollar denominated preferred securities, including functionally equivalent instruments, issued by U.S. financial companies. PGF provides a concentrated portfolio exclusively focused on financial-sector preferred securities and is considered non-diversified, holding both investment- and non-investment-grade securities within this focus.</code> | <code>The Allstate Corporation, together with its subsidiaries, provides property and casualty, and other insurance products in the United States and Canada. The company operates through Allstate Protection; Protection Services; Allstate Health and Benefits; and Run-off Property-Liability segments. The Allstate Protection segment offers private passenger auto and homeowners insurance; other personal lines products; and commercial lines products under the Allstate and Encompass brand names. The Protection Services segment provides consumer product protection plans and related technical support for mobile phones, consumer electronics, furniture, and appliances; finance and insurance products, including vehicle service contracts, guaranteed asset protection waivers, road hazard tire and wheel, and paint and fabric protection; towing, jump-start, lockout, fuel delivery, and tire change services; device and mobile data collection services; data and analytic solutions using automotive telematics i...</code> |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### stage2-dataset
+
+* Dataset: [stage2-dataset](https://huggingface.co/datasets/hobbang/stage2-dataset) at [cd393c2](https://huggingface.co/datasets/hobbang/stage2-dataset/tree/cd393c24f4017971e95aa6f73736f2fcb45e30a0)
+* Size: 16,944 evaluation samples
+* Columns: <code>anchor</code> and <code>positive</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                                | positive                                                                             |
+  |:--------|:--------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                                | string                                                                               |
+  | details | <ul><li>min: 135 tokens</li><li>mean: 149.21 tokens</li><li>max: 214 tokens</li></ul> | <ul><li>min: 42 tokens</li><li>mean: 252.75 tokens</li><li>max: 384 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>The Global X S&P 500 Risk Managed Income ETF seeks to track the Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets in index securities. The index's strategy involves holding the underlying stocks of the S&P 500 Index while applying an options collar, specifically selling at-the-money covered call options and buying monthly 5% out-of-the-money put options corresponding to the portfolio's value. This approach aims to generate income, ideally resulting in a net credit from the options premiums, and provide risk management, though selling at-the-money calls inherently caps the fund's potential for upside participation.</code> | <code>Apple Inc. designs, manufactures, and markets smartphones, personal computers, tablets, wearables, and accessories worldwide. The company offers iPhone, a line of smartphones; Mac, a line of personal computers; iPad, a line of multi-purpose tablets; and wearables, home, and accessories comprising AirPods, Apple TV, Apple Watch, Beats products, and HomePod. It also provides AppleCare support and cloud services; and operates various platforms, including the App Store that allow customers to discover and download applications and digital content, such as books, music, video, games, and podcasts, as well as advertising services include third-party licensing arrangements and its own advertising platforms. In addition, the company offers various subscription-based services, such as Apple Arcade, a game subscription service; Apple Fitness+, a personalized fitness service; Apple Music, which offers users a curated listening experience with on-demand radio stations; Apple News+, a subscription ...</code> |
+  | <code>The Global X S&P 500 Risk Managed Income ETF seeks to track the Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets in index securities. The index's strategy involves holding the underlying stocks of the S&P 500 Index while applying an options collar, specifically selling at-the-money covered call options and buying monthly 5% out-of-the-money put options corresponding to the portfolio's value. This approach aims to generate income, ideally resulting in a net credit from the options premiums, and provide risk management, though selling at-the-money calls inherently caps the fund's potential for upside participation.</code> | <code>Microsoft Corporation develops, licenses, and supports software, services, devices, and solutions worldwide. The company operates in three segments: Productivity and Business Processes, Intelligent Cloud, and More Personal Computing. The Productivity and Business Processes segment offers Office, Exchange, SharePoint, Microsoft Teams, Office 365 Security and Compliance, Microsoft Viva, and Skype for Business; Skype, Outlook.com, OneDrive, and LinkedIn; and Dynamics 365, a set of cloud-based and on-premises business solutions for organizations and enterprise divisions. The Intelligent Cloud segment licenses SQL, Windows Servers, Visual Studio, System Center, and related Client Access Licenses; GitHub that provides a collaboration platform and code hosting service for developers; Nuance provides healthcare and enterprise AI solutions; and Azure, a cloud platform. It also offers enterprise support, Microsoft consulting, and nuance professional services to assist customers in developing, de...</code> |
+  | <code>The Global X S&P 500 Risk Managed Income ETF seeks to track the Cboe S&P 500 Risk Managed Income Index by investing at least 80% of its assets in index securities. The index's strategy involves holding the underlying stocks of the S&P 500 Index while applying an options collar, specifically selling at-the-money covered call options and buying monthly 5% out-of-the-money put options corresponding to the portfolio's value. This approach aims to generate income, ideally resulting in a net credit from the options premiums, and provide risk management, though selling at-the-money calls inherently caps the fund's potential for upside participation.</code> | <code>NVIDIA Corporation provides graphics, and compute and networking solutions in the United States, Taiwan, China, and internationally. The company's Graphics segment offers GeForce GPUs for gaming and PCs, the GeForce NOW game streaming service and related infrastructure, and solutions for gaming platforms; Quadro/NVIDIA RTX GPUs for enterprise workstation graphics; vGPU software for cloud-based visual and virtual computing; automotive platforms for infotainment systems; and Omniverse software for building 3D designs and virtual worlds. Its Compute & Networking segment provides Data Center platforms and systems for AI, HPC, and accelerated computing; Mellanox networking and interconnect solutions; automotive AI Cockpit, autonomous driving development agreements, and autonomous vehicle solutions; cryptocurrency mining processors; Jetson for robotics and other embedded platforms; and NVIDIA AI Enterprise and other software. The company's products are used in gaming, professional visualizat...</code> |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 64
+- `per_device_eval_batch_size`: 32
+- `learning_rate`: 3e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `bf16`: True
+- `dataloader_drop_last`: True
+- `load_best_model_at_end`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 64
+- `per_device_eval_batch_size`: 32
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 3e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: True
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `tp_size`: 0
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0050 | 10   | 4.6656        | -               |
+| 0.0099 | 20   | 4.4733        | -               |
+| 0.0149 | 30   | 4.0093        | -               |
+| 0.0199 | 40   | 3.9259        | -               |
+| 0.0248 | 50   | 3.8315        | -               |
+| 0.0298 | 60   | 3.673         | -               |
+| 0.0347 | 70   | 3.5076        | -               |
+| 0.0397 | 80   | 3.4416        | -               |
+| 0.0447 | 90   | 3.4362        | -               |
+| 0.0496 | 100  | 3.3934        | -               |
+| 0.0546 | 110  | 3.3343        | -               |
+| 0.0596 | 120  | 3.3018        | -               |
+| 0.0645 | 130  | 3.2882        | -               |
+| 0.0695 | 140  | 3.3027        | -               |
+| 0.0744 | 150  | 3.2177        | -               |
+| 0.0794 | 160  | 3.2708        | -               |
+| 0.0844 | 170  | 3.2645        | -               |
+| 0.0893 | 180  | 3.1939        | -               |
+| 0.0943 | 190  | 3.0575        | -               |
+| 0.0993 | 200  | 3.0799        | -               |
+| 0.1042 | 210  | 3.0824        | -               |
+| 0.1092 | 220  | 3.0693        | -               |
+| 0.1141 | 230  | 3.1014        | -               |
+| 0.1191 | 240  | 3.0458        | -               |
+| 0.1241 | 250  | 3.04          | -               |
+| 0.1290 | 260  | 3.0311        | -               |
+| 0.1340 | 270  | 2.9778        | -               |
+| 0.1390 | 280  | 3.0701        | -               |
+| 0.1439 | 290  | 2.9039        | -               |
+| 0.1489 | 300  | 3.0449        | 2.5685          |
+| 0.1538 | 310  | 2.8896        | -               |
+| 0.1588 | 320  | 3.0527        | -               |
+| 0.1638 | 330  | 3.0153        | -               |
+| 0.1687 | 340  | 2.869         | -               |
+| 0.1737 | 350  | 2.9678        | -               |
+| 0.1787 | 360  | 2.9756        | -               |
+| 0.1836 | 370  | 2.9348        | -               |
+| 0.1886 | 380  | 2.9967        | -               |
+| 0.1935 | 390  | 2.8953        | -               |
+| 0.1985 | 400  | 2.9546        | -               |
+| 0.2035 | 410  | 2.9919        | -               |
+| 0.2084 | 420  | 2.8487        | -               |
+| 0.2134 | 430  | 2.7609        | -               |
+| 0.2184 | 440  | 2.9126        | -               |
+| 0.2233 | 450  | 2.8991        | -               |
+| 0.2283 | 460  | 2.9272        | -               |
+| 0.2333 | 470  | 2.9084        | -               |
+| 0.2382 | 480  | 2.7963        | -               |
+| 0.2432 | 490  | 2.822         | -               |
+| 0.2481 | 500  | 2.9376        | -               |
+| 0.2531 | 510  | 2.8969        | -               |
+| 0.2581 | 520  | 2.7745        | -               |
+| 0.2630 | 530  | 2.8103        | -               |
+| 0.2680 | 540  | 2.8189        | -               |
+| 0.2730 | 550  | 2.8322        | -               |
+| 0.2779 | 560  | 2.7627        | -               |
+| 0.2829 | 570  | 2.7796        | -               |
+| 0.2878 | 580  | 2.8515        | -               |
+| 0.2928 | 590  | 2.8758        | -               |
+| 0.2978 | 600  | 2.7963        | 2.4142          |
+| 0.3027 | 610  | 2.8259        | -               |
+| 0.3077 | 620  | 2.829         | -               |
+| 0.3127 | 630  | 2.7699        | -               |
+| 0.3176 | 640  | 2.7311        | -               |
+| 0.3226 | 650  | 2.735         | -               |
+| 0.3275 | 660  | 2.7306        | -               |
+| 0.3325 | 670  | 2.7467        | -               |
+| 0.3375 | 680  | 2.7494        | -               |
+| 0.3424 | 690  | 2.7386        | -               |
+| 0.3474 | 700  | 2.8513        | -               |
+| 0.3524 | 710  | 2.673         | -               |
+| 0.3573 | 720  | 2.8101        | -               |
+| 0.3623 | 730  | 2.7527        | -               |
+| 0.3672 | 740  | 2.7213        | -               |
+| 0.3722 | 750  | 2.753         | -               |
+| 0.3772 | 760  | 2.8034        | -               |
+| 0.3821 | 770  | 2.8288        | -               |
+| 0.3871 | 780  | 2.613         | -               |
+| 0.3921 | 790  | 2.7315        | -               |
+| 0.3970 | 800  | 2.8077        | -               |
+| 0.4020 | 810  | 2.7442        | -               |
+| 0.4069 | 820  | 2.7351        | -               |
+| 0.4119 | 830  | 2.7643        | -               |
+| 0.4169 | 840  | 2.8984        | -               |
+| 0.4218 | 850  | 2.7377        | -               |
+| 0.4268 | 860  | 2.7021        | -               |
+| 0.4318 | 870  | 2.6756        | -               |
+| 0.4367 | 880  | 2.7852        | -               |
+| 0.4417 | 890  | 2.7531        | -               |
+| 0.4467 | 900  | 2.6636        | 2.3456          |
+| 0.4516 | 910  | 2.7089        | -               |
+| 0.4566 | 920  | 2.8029        | -               |
+| 0.4615 | 930  | 2.721         | -               |
+| 0.4665 | 940  | 2.5606        | -               |
+| 0.4715 | 950  | 2.6397        | -               |
+| 0.4764 | 960  | 2.6563        | -               |
+| 0.4814 | 970  | 2.7163        | -               |
+| 0.4864 | 980  | 2.6225        | -               |
+| 0.4913 | 990  | 2.645         | -               |
+| 0.4963 | 1000 | 2.6576        | -               |
+| 0.5012 | 1010 | 2.7019        | -               |
+| 0.5062 | 1020 | 2.7195        | -               |
+| 0.5112 | 1030 | 2.7242        | -               |
+| 0.5161 | 1040 | 2.6729        | -               |
+| 0.5211 | 1050 | 2.7637        | -               |
+| 0.5261 | 1060 | 2.677         | -               |
+| 0.5310 | 1070 | 2.7018        | -               |
+| 0.5360 | 1080 | 2.6469        | -               |
+| 0.5409 | 1090 | 2.7186        | -               |
+| 0.5459 | 1100 | 2.6728        | -               |
+| 0.5509 | 1110 | 2.6694        | -               |
+| 0.5558 | 1120 | 2.7839        | -               |
+| 0.5608 | 1130 | 2.5834        | -               |
+| 0.5658 | 1140 | 2.6905        | -               |
+| 0.5707 | 1150 | 2.7223        | -               |
+| 0.5757 | 1160 | 2.7235        | -               |
+| 0.5806 | 1170 | 2.636         | -               |
+| 0.5856 | 1180 | 2.6314        | -               |
+| 0.5906 | 1190 | 2.5941        | -               |
+| 0.5955 | 1200 | 2.7827        | 2.2911          |
+| 0.6005 | 1210 | 2.6104        | -               |
+| 0.6055 | 1220 | 2.6148        | -               |
+| 0.6104 | 1230 | 2.6355        | -               |
+| 0.6154 | 1240 | 2.6269        | -               |
+| 0.6203 | 1250 | 2.6003        | -               |
+| 0.6253 | 1260 | 2.6256        | -               |
+| 0.6303 | 1270 | 2.6326        | -               |
+| 0.6352 | 1280 | 2.681         | -               |
+| 0.6402 | 1290 | 2.5776        | -               |
+| 0.6452 | 1300 | 2.7528        | -               |
+| 0.6501 | 1310 | 2.6076        | -               |
+| 0.6551 | 1320 | 2.5784        | -               |
+| 0.6600 | 1330 | 2.6064        | -               |
+| 0.6650 | 1340 | 2.5757        | -               |
+| 0.6700 | 1350 | 2.5851        | -               |
+| 0.6749 | 1360 | 2.6007        | -               |
+| 0.6799 | 1370 | 2.5674        | -               |
+| 0.6849 | 1380 | 2.6984        | -               |
+| 0.6898 | 1390 | 2.6202        | -               |
+| 0.6948 | 1400 | 2.6729        | -               |
+| 0.6998 | 1410 | 2.6683        | -               |
+| 0.7047 | 1420 | 2.6355        | -               |
+| 0.7097 | 1430 | 2.6033        | -               |
+| 0.7146 | 1440 | 2.6834        | -               |
+| 0.7196 | 1450 | 2.6597        | -               |
+| 0.7246 | 1460 | 2.6298        | -               |
+| 0.7295 | 1470 | 2.6232        | -               |
+| 0.7345 | 1480 | 2.5672        | -               |
+| 0.7395 | 1490 | 2.5139        | -               |
+| 0.7444 | 1500 | 2.6248        | 2.3090          |
+| 0.7494 | 1510 | 2.6417        | -               |
+| 0.7543 | 1520 | 2.6197        | -               |
+| 0.7593 | 1530 | 2.6911        | -               |
+| 0.7643 | 1540 | 2.5542        | -               |
+| 0.7692 | 1550 | 2.6584        | -               |
+| 0.7742 | 1560 | 2.6182        | -               |
+| 0.7792 | 1570 | 2.6301        | -               |
+| 0.7841 | 1580 | 2.5629        | -               |
+| 0.7891 | 1590 | 2.5965        | -               |
+| 0.7940 | 1600 | 2.5722        | -               |
+| 0.7990 | 1610 | 2.5835        | -               |
+| 0.8040 | 1620 | 2.5901        | -               |
+| 0.8089 | 1630 | 2.6055        | -               |
+| 0.8139 | 1640 | 2.6019        | -               |
+| 0.8189 | 1650 | 2.6421        | -               |
+| 0.8238 | 1660 | 2.6049        | -               |
+| 0.8288 | 1670 | 2.5351        | -               |
+| 0.8337 | 1680 | 2.6158        | -               |
+| 0.8387 | 1690 | 2.5994        | -               |
+| 0.8437 | 1700 | 2.5816        | -               |
+| 0.8486 | 1710 | 2.5848        | -               |
+| 0.8536 | 1720 | 2.6138        | -               |
+| 0.8586 | 1730 | 2.5811        | -               |
+| 0.8635 | 1740 | 2.5933        | -               |
+| 0.8685 | 1750 | 2.5869        | -               |
+| 0.8734 | 1760 | 2.5464        | -               |
+| 0.8784 | 1770 | 2.6842        | -               |
+| 0.8834 | 1780 | 2.6312        | -               |
+| 0.8883 | 1790 | 2.5621        | -               |
+| 0.8933 | 1800 | 2.6103        | 2.2858          |
+
+</details>
+
+### Framework Versions
+- Python: 3.10.12
+- Sentence Transformers: 4.1.0
+- Transformers: 4.51.3
+- PyTorch: 2.1.0+cu118
+- Accelerate: 1.6.0
+- Datasets: 3.5.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,23 @@
+{
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "4.1.0",
+    "transformers": "4.51.3",
+    "pytorch": "2.1.0+cu118"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a82d71fb5c4db686344d46bd5f3b89c03175ea4388f90df4bed4961e6715f53
+size 437967672

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02801de9884e5ed0a20cd47b4a20c53e0e4061c76d612a1cba170e4b46e9b937
+size 871331770

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e953d565ab8d03d27a69e4a09113a526233508f1995ed065d8d184b59c250a61
+size 14244

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abcaaf39545a0471725d97cdeb8ce9b6da32d113c78097ef6a9f53753bf81f30
+size 1064

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 384,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,73 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "max_length": 128,
+  "model_max_length": 384,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "</s>",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

trainer_state.json

@@ -0,0 +1,1351 @@
+{
+  "best_global_step": 1800,
+  "best_metric": 2.285778284072876,
+  "best_model_checkpoint": "models/mini_stage2/checkpoint-1800",
+  "epoch": 0.8933002481389578,
+  "eval_steps": 300,
+  "global_step": 1800,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.004962779156327543,
+      "grad_norm": 7.886381149291992,
+      "learning_rate": 1.3366336633663365e-06,
+      "loss": 4.6656,
+      "step": 10
+    },
+    {
+      "epoch": 0.009925558312655087,
+      "grad_norm": 5.6866021156311035,
+      "learning_rate": 2.821782178217822e-06,
+      "loss": 4.4733,
+      "step": 20
+    },
+    {
+      "epoch": 0.01488833746898263,
+      "grad_norm": 4.5250091552734375,
+      "learning_rate": 4.306930693069306e-06,
+      "loss": 4.0093,
+      "step": 30
+    },
+    {
+      "epoch": 0.019851116625310174,
+      "grad_norm": 3.3117995262145996,
+      "learning_rate": 5.792079207920792e-06,
+      "loss": 3.9259,
+      "step": 40
+    },
+    {
+      "epoch": 0.02481389578163772,
+      "grad_norm": 3.5785279273986816,
+      "learning_rate": 7.277227722772277e-06,
+      "loss": 3.8315,
+      "step": 50
+    },
+    {
+      "epoch": 0.02977667493796526,
+      "grad_norm": 5.32008695602417,
+      "learning_rate": 8.762376237623762e-06,
+      "loss": 3.673,
+      "step": 60
+    },
+    {
+      "epoch": 0.034739454094292806,
+      "grad_norm": 4.776355266571045,
+      "learning_rate": 1.0247524752475248e-05,
+      "loss": 3.5076,
+      "step": 70
+    },
+    {
+      "epoch": 0.03970223325062035,
+      "grad_norm": 4.523674488067627,
+      "learning_rate": 1.1732673267326734e-05,
+      "loss": 3.4416,
+      "step": 80
+    },
+    {
+      "epoch": 0.04466501240694789,
+      "grad_norm": 4.444321155548096,
+      "learning_rate": 1.3217821782178218e-05,
+      "loss": 3.4362,
+      "step": 90
+    },
+    {
+      "epoch": 0.04962779156327544,
+      "grad_norm": 4.364451885223389,
+      "learning_rate": 1.4702970297029704e-05,
+      "loss": 3.3934,
+      "step": 100
+    },
+    {
+      "epoch": 0.05459057071960298,
+      "grad_norm": 5.098319053649902,
+      "learning_rate": 1.618811881188119e-05,
+      "loss": 3.3343,
+      "step": 110
+    },
+    {
+      "epoch": 0.05955334987593052,
+      "grad_norm": 4.762153148651123,
+      "learning_rate": 1.7673267326732672e-05,
+      "loss": 3.3018,
+      "step": 120
+    },
+    {
+      "epoch": 0.06451612903225806,
+      "grad_norm": 4.8034563064575195,
+      "learning_rate": 1.915841584158416e-05,
+      "loss": 3.2882,
+      "step": 130
+    },
+    {
+      "epoch": 0.06947890818858561,
+      "grad_norm": 4.955429553985596,
+      "learning_rate": 2.0643564356435643e-05,
+      "loss": 3.3027,
+      "step": 140
+    },
+    {
+      "epoch": 0.07444168734491315,
+      "grad_norm": 4.338839530944824,
+      "learning_rate": 2.212871287128713e-05,
+      "loss": 3.2177,
+      "step": 150
+    },
+    {
+      "epoch": 0.0794044665012407,
+      "grad_norm": 4.854152202606201,
+      "learning_rate": 2.3613861386138615e-05,
+      "loss": 3.2708,
+      "step": 160
+    },
+    {
+      "epoch": 0.08436724565756824,
+      "grad_norm": 4.855740070343018,
+      "learning_rate": 2.5099009900990097e-05,
+      "loss": 3.2645,
+      "step": 170
+    },
+    {
+      "epoch": 0.08933002481389578,
+      "grad_norm": 5.82074499130249,
+      "learning_rate": 2.6584158415841586e-05,
+      "loss": 3.1939,
+      "step": 180
+    },
+    {
+      "epoch": 0.09429280397022333,
+      "grad_norm": 5.284363746643066,
+      "learning_rate": 2.8069306930693072e-05,
+      "loss": 3.0575,
+      "step": 190
+    },
+    {
+      "epoch": 0.09925558312655088,
+      "grad_norm": 5.663958549499512,
+      "learning_rate": 2.9554455445544555e-05,
+      "loss": 3.0799,
+      "step": 200
+    },
+    {
+      "epoch": 0.10421836228287841,
+      "grad_norm": 5.67520809173584,
+      "learning_rate": 2.9884169884169887e-05,
+      "loss": 3.0824,
+      "step": 210
+    },
+    {
+      "epoch": 0.10918114143920596,
+      "grad_norm": 5.432056903839111,
+      "learning_rate": 2.9718698290126862e-05,
+      "loss": 3.0693,
+      "step": 220
+    },
+    {
+      "epoch": 0.1141439205955335,
+      "grad_norm": 5.387454032897949,
+      "learning_rate": 2.9553226696083837e-05,
+      "loss": 3.1014,
+      "step": 230
+    },
+    {
+      "epoch": 0.11910669975186104,
+      "grad_norm": 5.515647888183594,
+      "learning_rate": 2.9387755102040816e-05,
+      "loss": 3.0458,
+      "step": 240
+    },
+    {
+      "epoch": 0.12406947890818859,
+      "grad_norm": 5.962894916534424,
+      "learning_rate": 2.9222283507997795e-05,
+      "loss": 3.04,
+      "step": 250
+    },
+    {
+      "epoch": 0.12903225806451613,
+      "grad_norm": 5.549206733703613,
+      "learning_rate": 2.9056811913954774e-05,
+      "loss": 3.0311,
+      "step": 260
+    },
+    {
+      "epoch": 0.13399503722084366,
+      "grad_norm": 4.67368221282959,
+      "learning_rate": 2.889134031991175e-05,
+      "loss": 2.9778,
+      "step": 270
+    },
+    {
+      "epoch": 0.13895781637717122,
+      "grad_norm": 5.445274829864502,
+      "learning_rate": 2.8725868725868724e-05,
+      "loss": 3.0701,
+      "step": 280
+    },
+    {
+      "epoch": 0.14392059553349876,
+      "grad_norm": 5.643553256988525,
+      "learning_rate": 2.8560397131825703e-05,
+      "loss": 2.9039,
+      "step": 290
+    },
+    {
+      "epoch": 0.1488833746898263,
+      "grad_norm": 5.453239440917969,
+      "learning_rate": 2.8394925537782682e-05,
+      "loss": 3.0449,
+      "step": 300
+    },
+    {
+      "epoch": 0.1488833746898263,
+      "eval_loss": 2.5684616565704346,
+      "eval_runtime": 119.1131,
+      "eval_samples_per_second": 142.251,
+      "eval_steps_per_second": 4.45,
+      "step": 300
+    },
+    {
+      "epoch": 0.15384615384615385,
+      "grad_norm": 5.800920009613037,
+      "learning_rate": 2.822945394373966e-05,
+      "loss": 2.8896,
+      "step": 310
+    },
+    {
+      "epoch": 0.1588089330024814,
+      "grad_norm": 4.8736724853515625,
+      "learning_rate": 2.8063982349696636e-05,
+      "loss": 3.0527,
+      "step": 320
+    },
+    {
+      "epoch": 0.16377171215880892,
+      "grad_norm": 4.498941898345947,
+      "learning_rate": 2.7898510755653615e-05,
+      "loss": 3.0153,
+      "step": 330
+    },
+    {
+      "epoch": 0.1687344913151365,
+      "grad_norm": 5.518968105316162,
+      "learning_rate": 2.773303916161059e-05,
+      "loss": 2.869,
+      "step": 340
+    },
+    {
+      "epoch": 0.17369727047146402,
+      "grad_norm": 5.431845188140869,
+      "learning_rate": 2.756756756756757e-05,
+      "loss": 2.9678,
+      "step": 350
+    },
+    {
+      "epoch": 0.17866004962779156,
+      "grad_norm": 5.1342973709106445,
+      "learning_rate": 2.7402095973524544e-05,
+      "loss": 2.9756,
+      "step": 360
+    },
+    {
+      "epoch": 0.18362282878411912,
+      "grad_norm": 4.8498101234436035,
+      "learning_rate": 2.7236624379481523e-05,
+      "loss": 2.9348,
+      "step": 370
+    },
+    {
+      "epoch": 0.18858560794044665,
+      "grad_norm": 5.223404407501221,
+      "learning_rate": 2.70711527854385e-05,
+      "loss": 2.9967,
+      "step": 380
+    },
+    {
+      "epoch": 0.1935483870967742,
+      "grad_norm": 5.078680038452148,
+      "learning_rate": 2.690568119139548e-05,
+      "loss": 2.8953,
+      "step": 390
+    },
+    {
+      "epoch": 0.19851116625310175,
+      "grad_norm": 4.336000442504883,
+      "learning_rate": 2.6740209597352456e-05,
+      "loss": 2.9546,
+      "step": 400
+    },
+    {
+      "epoch": 0.20347394540942929,
+      "grad_norm": 5.194223880767822,
+      "learning_rate": 2.657473800330943e-05,
+      "loss": 2.9919,
+      "step": 410
+    },
+    {
+      "epoch": 0.20843672456575682,
+      "grad_norm": 4.870718002319336,
+      "learning_rate": 2.640926640926641e-05,
+      "loss": 2.8487,
+      "step": 420
+    },
+    {
+      "epoch": 0.21339950372208435,
+      "grad_norm": 5.526561260223389,
+      "learning_rate": 2.624379481522339e-05,
+      "loss": 2.7609,
+      "step": 430
+    },
+    {
+      "epoch": 0.21836228287841192,
+      "grad_norm": 5.372407913208008,
+      "learning_rate": 2.6078323221180364e-05,
+      "loss": 2.9126,
+      "step": 440
+    },
+    {
+      "epoch": 0.22332506203473945,
+      "grad_norm": 4.934298992156982,
+      "learning_rate": 2.5912851627137343e-05,
+      "loss": 2.8991,
+      "step": 450
+    },
+    {
+      "epoch": 0.228287841191067,
+      "grad_norm": 4.916905403137207,
+      "learning_rate": 2.574738003309432e-05,
+      "loss": 2.9272,
+      "step": 460
+    },
+    {
+      "epoch": 0.23325062034739455,
+      "grad_norm": 5.660480976104736,
+      "learning_rate": 2.5581908439051297e-05,
+      "loss": 2.9084,
+      "step": 470
+    },
+    {
+      "epoch": 0.23821339950372208,
+      "grad_norm": 4.987634658813477,
+      "learning_rate": 2.5416436845008272e-05,
+      "loss": 2.7963,
+      "step": 480
+    },
+    {
+      "epoch": 0.24317617866004962,
+      "grad_norm": 6.746623992919922,
+      "learning_rate": 2.525096525096525e-05,
+      "loss": 2.822,
+      "step": 490
+    },
+    {
+      "epoch": 0.24813895781637718,
+      "grad_norm": 4.658477783203125,
+      "learning_rate": 2.508549365692223e-05,
+      "loss": 2.9376,
+      "step": 500
+    },
+    {
+      "epoch": 0.2531017369727047,
+      "grad_norm": 5.2974162101745605,
+      "learning_rate": 2.492002206287921e-05,
+      "loss": 2.8969,
+      "step": 510
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 5.682817459106445,
+      "learning_rate": 2.4754550468836184e-05,
+      "loss": 2.7745,
+      "step": 520
+    },
+    {
+      "epoch": 0.2630272952853598,
+      "grad_norm": 5.85469388961792,
+      "learning_rate": 2.458907887479316e-05,
+      "loss": 2.8103,
+      "step": 530
+    },
+    {
+      "epoch": 0.2679900744416873,
+      "grad_norm": 4.600194454193115,
+      "learning_rate": 2.4423607280750138e-05,
+      "loss": 2.8189,
+      "step": 540
+    },
+    {
+      "epoch": 0.2729528535980149,
+      "grad_norm": 6.168639659881592,
+      "learning_rate": 2.4258135686707117e-05,
+      "loss": 2.8322,
+      "step": 550
+    },
+    {
+      "epoch": 0.27791563275434245,
+      "grad_norm": 4.763863563537598,
+      "learning_rate": 2.4092664092664092e-05,
+      "loss": 2.7627,
+      "step": 560
+    },
+    {
+      "epoch": 0.28287841191067,
+      "grad_norm": 5.0977091789245605,
+      "learning_rate": 2.392719249862107e-05,
+      "loss": 2.7796,
+      "step": 570
+    },
+    {
+      "epoch": 0.2878411910669975,
+      "grad_norm": 4.786524772644043,
+      "learning_rate": 2.376172090457805e-05,
+      "loss": 2.8515,
+      "step": 580
+    },
+    {
+      "epoch": 0.29280397022332505,
+      "grad_norm": 5.386171817779541,
+      "learning_rate": 2.3596249310535025e-05,
+      "loss": 2.8758,
+      "step": 590
+    },
+    {
+      "epoch": 0.2977667493796526,
+      "grad_norm": 4.96185827255249,
+      "learning_rate": 2.3430777716492e-05,
+      "loss": 2.7963,
+      "step": 600
+    },
+    {
+      "epoch": 0.2977667493796526,
+      "eval_loss": 2.4141688346862793,
+      "eval_runtime": 112.2744,
+      "eval_samples_per_second": 150.916,
+      "eval_steps_per_second": 4.721,
+      "step": 600
+    },
+    {
+      "epoch": 0.3027295285359802,
+      "grad_norm": 7.280220985412598,
+      "learning_rate": 2.326530612244898e-05,
+      "loss": 2.8259,
+      "step": 610
+    },
+    {
+      "epoch": 0.3076923076923077,
+      "grad_norm": 5.671541690826416,
+      "learning_rate": 2.3099834528405958e-05,
+      "loss": 2.829,
+      "step": 620
+    },
+    {
+      "epoch": 0.31265508684863524,
+      "grad_norm": 5.268543720245361,
+      "learning_rate": 2.2934362934362936e-05,
+      "loss": 2.7699,
+      "step": 630
+    },
+    {
+      "epoch": 0.3176178660049628,
+      "grad_norm": 4.945257186889648,
+      "learning_rate": 2.2768891340319915e-05,
+      "loss": 2.7311,
+      "step": 640
+    },
+    {
+      "epoch": 0.3225806451612903,
+      "grad_norm": 5.198265075683594,
+      "learning_rate": 2.2603419746276887e-05,
+      "loss": 2.735,
+      "step": 650
+    },
+    {
+      "epoch": 0.32754342431761785,
+      "grad_norm": 4.624203205108643,
+      "learning_rate": 2.2437948152233866e-05,
+      "loss": 2.7306,
+      "step": 660
+    },
+    {
+      "epoch": 0.3325062034739454,
+      "grad_norm": 5.143148899078369,
+      "learning_rate": 2.2272476558190845e-05,
+      "loss": 2.7467,
+      "step": 670
+    },
+    {
+      "epoch": 0.337468982630273,
+      "grad_norm": 4.643360614776611,
+      "learning_rate": 2.2107004964147823e-05,
+      "loss": 2.7494,
+      "step": 680
+    },
+    {
+      "epoch": 0.3424317617866005,
+      "grad_norm": 5.404439926147461,
+      "learning_rate": 2.19415333701048e-05,
+      "loss": 2.7386,
+      "step": 690
+    },
+    {
+      "epoch": 0.34739454094292804,
+      "grad_norm": 4.590571403503418,
+      "learning_rate": 2.1776061776061778e-05,
+      "loss": 2.8513,
+      "step": 700
+    },
+    {
+      "epoch": 0.3523573200992556,
+      "grad_norm": 5.185126781463623,
+      "learning_rate": 2.1610590182018756e-05,
+      "loss": 2.673,
+      "step": 710
+    },
+    {
+      "epoch": 0.3573200992555831,
+      "grad_norm": 5.4743547439575195,
+      "learning_rate": 2.144511858797573e-05,
+      "loss": 2.8101,
+      "step": 720
+    },
+    {
+      "epoch": 0.36228287841191065,
+      "grad_norm": 5.463687896728516,
+      "learning_rate": 2.1279646993932707e-05,
+      "loss": 2.7527,
+      "step": 730
+    },
+    {
+      "epoch": 0.36724565756823824,
+      "grad_norm": 6.102997303009033,
+      "learning_rate": 2.1114175399889686e-05,
+      "loss": 2.7213,
+      "step": 740
+    },
+    {
+      "epoch": 0.37220843672456577,
+      "grad_norm": 5.1998724937438965,
+      "learning_rate": 2.0948703805846664e-05,
+      "loss": 2.753,
+      "step": 750
+    },
+    {
+      "epoch": 0.3771712158808933,
+      "grad_norm": 5.181605815887451,
+      "learning_rate": 2.0783232211803643e-05,
+      "loss": 2.8034,
+      "step": 760
+    },
+    {
+      "epoch": 0.38213399503722084,
+      "grad_norm": 5.865420818328857,
+      "learning_rate": 2.061776061776062e-05,
+      "loss": 2.8288,
+      "step": 770
+    },
+    {
+      "epoch": 0.3870967741935484,
+      "grad_norm": 4.965085029602051,
+      "learning_rate": 2.0452289023717594e-05,
+      "loss": 2.613,
+      "step": 780
+    },
+    {
+      "epoch": 0.3920595533498759,
+      "grad_norm": 5.534601211547852,
+      "learning_rate": 2.0286817429674573e-05,
+      "loss": 2.7315,
+      "step": 790
+    },
+    {
+      "epoch": 0.3970223325062035,
+      "grad_norm": 5.220632076263428,
+      "learning_rate": 2.012134583563155e-05,
+      "loss": 2.8077,
+      "step": 800
+    },
+    {
+      "epoch": 0.40198511166253104,
+      "grad_norm": 5.059537410736084,
+      "learning_rate": 1.9955874241588527e-05,
+      "loss": 2.7442,
+      "step": 810
+    },
+    {
+      "epoch": 0.40694789081885857,
+      "grad_norm": 5.325795650482178,
+      "learning_rate": 1.9790402647545506e-05,
+      "loss": 2.7351,
+      "step": 820
+    },
+    {
+      "epoch": 0.4119106699751861,
+      "grad_norm": 8.455648422241211,
+      "learning_rate": 1.9624931053502484e-05,
+      "loss": 2.7643,
+      "step": 830
+    },
+    {
+      "epoch": 0.41687344913151364,
+      "grad_norm": 5.344241619110107,
+      "learning_rate": 1.945945945945946e-05,
+      "loss": 2.8984,
+      "step": 840
+    },
+    {
+      "epoch": 0.4218362282878412,
+      "grad_norm": 4.816617965698242,
+      "learning_rate": 1.9293987865416435e-05,
+      "loss": 2.7377,
+      "step": 850
+    },
+    {
+      "epoch": 0.4267990074441687,
+      "grad_norm": 5.334592819213867,
+      "learning_rate": 1.9128516271373414e-05,
+      "loss": 2.7021,
+      "step": 860
+    },
+    {
+      "epoch": 0.4317617866004963,
+      "grad_norm": 5.0712385177612305,
+      "learning_rate": 1.8963044677330393e-05,
+      "loss": 2.6756,
+      "step": 870
+    },
+    {
+      "epoch": 0.43672456575682383,
+      "grad_norm": 5.225603103637695,
+      "learning_rate": 1.879757308328737e-05,
+      "loss": 2.7852,
+      "step": 880
+    },
+    {
+      "epoch": 0.44168734491315137,
+      "grad_norm": 5.172723770141602,
+      "learning_rate": 1.8632101489244347e-05,
+      "loss": 2.7531,
+      "step": 890
+    },
+    {
+      "epoch": 0.4466501240694789,
+      "grad_norm": 4.887333869934082,
+      "learning_rate": 1.8466629895201322e-05,
+      "loss": 2.6636,
+      "step": 900
+    },
+    {
+      "epoch": 0.4466501240694789,
+      "eval_loss": 2.3456013202667236,
+      "eval_runtime": 113.8402,
+      "eval_samples_per_second": 148.84,
+      "eval_steps_per_second": 4.656,
+      "step": 900
+    },
+    {
+      "epoch": 0.45161290322580644,
+      "grad_norm": 5.249894142150879,
+      "learning_rate": 1.83011583011583e-05,
+      "loss": 2.7089,
+      "step": 910
+    },
+    {
+      "epoch": 0.456575682382134,
+      "grad_norm": 4.984340667724609,
+      "learning_rate": 1.813568670711528e-05,
+      "loss": 2.8029,
+      "step": 920
+    },
+    {
+      "epoch": 0.46153846153846156,
+      "grad_norm": 5.297105312347412,
+      "learning_rate": 1.7970215113072255e-05,
+      "loss": 2.721,
+      "step": 930
+    },
+    {
+      "epoch": 0.4665012406947891,
+      "grad_norm": 5.074361801147461,
+      "learning_rate": 1.7804743519029234e-05,
+      "loss": 2.5606,
+      "step": 940
+    },
+    {
+      "epoch": 0.47146401985111663,
+      "grad_norm": 5.639915943145752,
+      "learning_rate": 1.7639271924986212e-05,
+      "loss": 2.6397,
+      "step": 950
+    },
+    {
+      "epoch": 0.47642679900744417,
+      "grad_norm": 5.445567607879639,
+      "learning_rate": 1.7473800330943188e-05,
+      "loss": 2.6563,
+      "step": 960
+    },
+    {
+      "epoch": 0.4813895781637717,
+      "grad_norm": 4.462475299835205,
+      "learning_rate": 1.7308328736900166e-05,
+      "loss": 2.7163,
+      "step": 970
+    },
+    {
+      "epoch": 0.48635235732009924,
+      "grad_norm": 5.833073139190674,
+      "learning_rate": 1.7142857142857142e-05,
+      "loss": 2.6225,
+      "step": 980
+    },
+    {
+      "epoch": 0.4913151364764268,
+      "grad_norm": 5.428512096405029,
+      "learning_rate": 1.697738554881412e-05,
+      "loss": 2.645,
+      "step": 990
+    },
+    {
+      "epoch": 0.49627791563275436,
+      "grad_norm": 4.768510818481445,
+      "learning_rate": 1.68119139547711e-05,
+      "loss": 2.6576,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5012406947890818,
+      "grad_norm": 4.692178726196289,
+      "learning_rate": 1.6646442360728078e-05,
+      "loss": 2.7019,
+      "step": 1010
+    },
+    {
+      "epoch": 0.5062034739454094,
+      "grad_norm": 5.51165771484375,
+      "learning_rate": 1.6480970766685053e-05,
+      "loss": 2.7195,
+      "step": 1020
+    },
+    {
+      "epoch": 0.511166253101737,
+      "grad_norm": 4.9710516929626465,
+      "learning_rate": 1.631549917264203e-05,
+      "loss": 2.7242,
+      "step": 1030
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 5.166830062866211,
+      "learning_rate": 1.6150027578599008e-05,
+      "loss": 2.6729,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5210918114143921,
+      "grad_norm": 4.945868015289307,
+      "learning_rate": 1.5984555984555986e-05,
+      "loss": 2.7637,
+      "step": 1050
+    },
+    {
+      "epoch": 0.5260545905707196,
+      "grad_norm": 5.7991943359375,
+      "learning_rate": 1.581908439051296e-05,
+      "loss": 2.677,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5310173697270472,
+      "grad_norm": 5.255971431732178,
+      "learning_rate": 1.565361279646994e-05,
+      "loss": 2.7018,
+      "step": 1070
+    },
+    {
+      "epoch": 0.5359801488833746,
+      "grad_norm": 5.910732746124268,
+      "learning_rate": 1.548814120242692e-05,
+      "loss": 2.6469,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5409429280397022,
+      "grad_norm": 4.5612616539001465,
+      "learning_rate": 1.5322669608383895e-05,
+      "loss": 2.7186,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5459057071960298,
+      "grad_norm": 5.111081600189209,
+      "learning_rate": 1.515719801434087e-05,
+      "loss": 2.6728,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5508684863523573,
+      "grad_norm": 5.450465202331543,
+      "learning_rate": 1.4991726420297849e-05,
+      "loss": 2.6694,
+      "step": 1110
+    },
+    {
+      "epoch": 0.5558312655086849,
+      "grad_norm": 4.850940704345703,
+      "learning_rate": 1.4826254826254827e-05,
+      "loss": 2.7839,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5607940446650124,
+      "grad_norm": 5.036434173583984,
+      "learning_rate": 1.4660783232211803e-05,
+      "loss": 2.5834,
+      "step": 1130
+    },
+    {
+      "epoch": 0.56575682382134,
+      "grad_norm": 40.256370544433594,
+      "learning_rate": 1.4495311638168781e-05,
+      "loss": 2.6905,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5707196029776674,
+      "grad_norm": 5.939204692840576,
+      "learning_rate": 1.4329840044125759e-05,
+      "loss": 2.7223,
+      "step": 1150
+    },
+    {
+      "epoch": 0.575682382133995,
+      "grad_norm": 5.288106441497803,
+      "learning_rate": 1.4164368450082736e-05,
+      "loss": 2.7235,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5806451612903226,
+      "grad_norm": 5.540660858154297,
+      "learning_rate": 1.3998896856039713e-05,
+      "loss": 2.636,
+      "step": 1170
+    },
+    {
+      "epoch": 0.5856079404466501,
+      "grad_norm": 5.182808876037598,
+      "learning_rate": 1.3833425261996691e-05,
+      "loss": 2.6314,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5905707196029777,
+      "grad_norm": 4.9914140701293945,
+      "learning_rate": 1.3667953667953668e-05,
+      "loss": 2.5941,
+      "step": 1190
+    },
+    {
+      "epoch": 0.5955334987593052,
+      "grad_norm": 4.8229475021362305,
+      "learning_rate": 1.3502482073910646e-05,
+      "loss": 2.7827,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5955334987593052,
+      "eval_loss": 2.2910733222961426,
+      "eval_runtime": 115.8769,
+      "eval_samples_per_second": 146.224,
+      "eval_steps_per_second": 4.574,
+      "step": 1200
+    },
+    {
+      "epoch": 0.6004962779156328,
+      "grad_norm": 5.556408882141113,
+      "learning_rate": 1.3337010479867623e-05,
+      "loss": 2.6104,
+      "step": 1210
+    },
+    {
+      "epoch": 0.6054590570719603,
+      "grad_norm": 6.0620551109313965,
+      "learning_rate": 1.3171538885824601e-05,
+      "loss": 2.6148,
+      "step": 1220
+    },
+    {
+      "epoch": 0.6104218362282878,
+      "grad_norm": 5.285867214202881,
+      "learning_rate": 1.3006067291781577e-05,
+      "loss": 2.6355,
+      "step": 1230
+    },
+    {
+      "epoch": 0.6153846153846154,
+      "grad_norm": 5.499397277832031,
+      "learning_rate": 1.2840595697738555e-05,
+      "loss": 2.6269,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6203473945409429,
+      "grad_norm": 4.932219505310059,
+      "learning_rate": 1.2675124103695532e-05,
+      "loss": 2.6003,
+      "step": 1250
+    },
+    {
+      "epoch": 0.6253101736972705,
+      "grad_norm": 5.04417610168457,
+      "learning_rate": 1.250965250965251e-05,
+      "loss": 2.6256,
+      "step": 1260
+    },
+    {
+      "epoch": 0.630272952853598,
+      "grad_norm": 5.0753679275512695,
+      "learning_rate": 1.2344180915609488e-05,
+      "loss": 2.6326,
+      "step": 1270
+    },
+    {
+      "epoch": 0.6352357320099256,
+      "grad_norm": 5.003329753875732,
+      "learning_rate": 1.2178709321566465e-05,
+      "loss": 2.681,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6401985111662531,
+      "grad_norm": 5.094631671905518,
+      "learning_rate": 1.2013237727523442e-05,
+      "loss": 2.5776,
+      "step": 1290
+    },
+    {
+      "epoch": 0.6451612903225806,
+      "grad_norm": 4.915186405181885,
+      "learning_rate": 1.184776613348042e-05,
+      "loss": 2.7528,
+      "step": 1300
+    },
+    {
+      "epoch": 0.6501240694789082,
+      "grad_norm": 5.3234076499938965,
+      "learning_rate": 1.1682294539437398e-05,
+      "loss": 2.6076,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6550868486352357,
+      "grad_norm": 6.5297064781188965,
+      "learning_rate": 1.1516822945394374e-05,
+      "loss": 2.5784,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6600496277915633,
+      "grad_norm": 5.549312591552734,
+      "learning_rate": 1.1351351351351352e-05,
+      "loss": 2.6064,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6650124069478908,
+      "grad_norm": 4.589717388153076,
+      "learning_rate": 1.118587975730833e-05,
+      "loss": 2.5757,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6699751861042184,
+      "grad_norm": 6.217405796051025,
+      "learning_rate": 1.1020408163265306e-05,
+      "loss": 2.5851,
+      "step": 1350
+    },
+    {
+      "epoch": 0.674937965260546,
+      "grad_norm": 5.602436542510986,
+      "learning_rate": 1.0854936569222283e-05,
+      "loss": 2.6007,
+      "step": 1360
+    },
+    {
+      "epoch": 0.6799007444168734,
+      "grad_norm": 4.934968948364258,
+      "learning_rate": 1.0689464975179262e-05,
+      "loss": 2.5674,
+      "step": 1370
+    },
+    {
+      "epoch": 0.684863523573201,
+      "grad_norm": 5.8763322830200195,
+      "learning_rate": 1.0523993381136238e-05,
+      "loss": 2.6984,
+      "step": 1380
+    },
+    {
+      "epoch": 0.6898263027295285,
+      "grad_norm": 5.077223777770996,
+      "learning_rate": 1.0358521787093216e-05,
+      "loss": 2.6202,
+      "step": 1390
+    },
+    {
+      "epoch": 0.6947890818858561,
+      "grad_norm": 6.423431873321533,
+      "learning_rate": 1.0193050193050193e-05,
+      "loss": 2.6729,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6997518610421837,
+      "grad_norm": 5.683679103851318,
+      "learning_rate": 1.002757859900717e-05,
+      "loss": 2.6683,
+      "step": 1410
+    },
+    {
+      "epoch": 0.7047146401985112,
+      "grad_norm": 5.217991828918457,
+      "learning_rate": 9.862107004964148e-06,
+      "loss": 2.6355,
+      "step": 1420
+    },
+    {
+      "epoch": 0.7096774193548387,
+      "grad_norm": 5.0821332931518555,
+      "learning_rate": 9.696635410921126e-06,
+      "loss": 2.6033,
+      "step": 1430
+    },
+    {
+      "epoch": 0.7146401985111662,
+      "grad_norm": 5.337713718414307,
+      "learning_rate": 9.531163816878102e-06,
+      "loss": 2.6834,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7196029776674938,
+      "grad_norm": 5.425897598266602,
+      "learning_rate": 9.36569222283508e-06,
+      "loss": 2.6597,
+      "step": 1450
+    },
+    {
+      "epoch": 0.7245657568238213,
+      "grad_norm": 4.878344535827637,
+      "learning_rate": 9.200220628792057e-06,
+      "loss": 2.6298,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7295285359801489,
+      "grad_norm": 5.754893779754639,
+      "learning_rate": 9.034749034749034e-06,
+      "loss": 2.6232,
+      "step": 1470
+    },
+    {
+      "epoch": 0.7344913151364765,
+      "grad_norm": 4.991476535797119,
+      "learning_rate": 8.869277440706012e-06,
+      "loss": 2.5672,
+      "step": 1480
+    },
+    {
+      "epoch": 0.739454094292804,
+      "grad_norm": 6.236905097961426,
+      "learning_rate": 8.70380584666299e-06,
+      "loss": 2.5139,
+      "step": 1490
+    },
+    {
+      "epoch": 0.7444168734491315,
+      "grad_norm": 4.892019748687744,
+      "learning_rate": 8.538334252619966e-06,
+      "loss": 2.6248,
+      "step": 1500
+    },
+    {
+      "epoch": 0.7444168734491315,
+      "eval_loss": 2.3090243339538574,
+      "eval_runtime": 114.529,
+      "eval_samples_per_second": 147.945,
+      "eval_steps_per_second": 4.628,
+      "step": 1500
+    },
+    {
+      "epoch": 0.749379652605459,
+      "grad_norm": 5.293705463409424,
+      "learning_rate": 8.372862658576944e-06,
+      "loss": 2.6417,
+      "step": 1510
+    },
+    {
+      "epoch": 0.7543424317617866,
+      "grad_norm": 6.500164031982422,
+      "learning_rate": 8.207391064533921e-06,
+      "loss": 2.6197,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7593052109181141,
+      "grad_norm": 5.456136226654053,
+      "learning_rate": 8.0419194704909e-06,
+      "loss": 2.6911,
+      "step": 1530
+    },
+    {
+      "epoch": 0.7642679900744417,
+      "grad_norm": 5.395550727844238,
+      "learning_rate": 7.876447876447876e-06,
+      "loss": 2.5542,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7692307692307693,
+      "grad_norm": 5.446476936340332,
+      "learning_rate": 7.710976282404854e-06,
+      "loss": 2.6584,
+      "step": 1550
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 5.699354648590088,
+      "learning_rate": 7.5455046883618305e-06,
+      "loss": 2.6182,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7791563275434243,
+      "grad_norm": 5.835291862487793,
+      "learning_rate": 7.3800330943188084e-06,
+      "loss": 2.6301,
+      "step": 1570
+    },
+    {
+      "epoch": 0.7841191066997518,
+      "grad_norm": 5.066339015960693,
+      "learning_rate": 7.214561500275786e-06,
+      "loss": 2.5629,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7890818858560794,
+      "grad_norm": 6.113178253173828,
+      "learning_rate": 7.049089906232763e-06,
+      "loss": 2.5965,
+      "step": 1590
+    },
+    {
+      "epoch": 0.794044665012407,
+      "grad_norm": 5.544576168060303,
+      "learning_rate": 6.8836183121897404e-06,
+      "loss": 2.5722,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7990074441687345,
+      "grad_norm": 5.560300350189209,
+      "learning_rate": 6.718146718146718e-06,
+      "loss": 2.5835,
+      "step": 1610
+    },
+    {
+      "epoch": 0.8039702233250621,
+      "grad_norm": 5.354392051696777,
+      "learning_rate": 6.552675124103695e-06,
+      "loss": 2.5901,
+      "step": 1620
+    },
+    {
+      "epoch": 0.8089330024813896,
+      "grad_norm": 5.256809234619141,
+      "learning_rate": 6.387203530060673e-06,
+      "loss": 2.6055,
+      "step": 1630
+    },
+    {
+      "epoch": 0.8138957816377171,
+      "grad_norm": 6.089339256286621,
+      "learning_rate": 6.22173193601765e-06,
+      "loss": 2.6019,
+      "step": 1640
+    },
+    {
+      "epoch": 0.8188585607940446,
+      "grad_norm": 4.713355541229248,
+      "learning_rate": 6.056260341974627e-06,
+      "loss": 2.6421,
+      "step": 1650
+    },
+    {
+      "epoch": 0.8238213399503722,
+      "grad_norm": 6.513923168182373,
+      "learning_rate": 5.890788747931605e-06,
+      "loss": 2.6049,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8287841191066998,
+      "grad_norm": 5.37587833404541,
+      "learning_rate": 5.725317153888582e-06,
+      "loss": 2.5351,
+      "step": 1670
+    },
+    {
+      "epoch": 0.8337468982630273,
+      "grad_norm": 6.346861362457275,
+      "learning_rate": 5.5598455598455594e-06,
+      "loss": 2.6158,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8387096774193549,
+      "grad_norm": 5.206553936004639,
+      "learning_rate": 5.394373965802537e-06,
+      "loss": 2.5994,
+      "step": 1690
+    },
+    {
+      "epoch": 0.8436724565756824,
+      "grad_norm": 6.567471504211426,
+      "learning_rate": 5.228902371759514e-06,
+      "loss": 2.5816,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8486352357320099,
+      "grad_norm": 5.689151287078857,
+      "learning_rate": 5.0634307777164914e-06,
+      "loss": 2.5848,
+      "step": 1710
+    },
+    {
+      "epoch": 0.8535980148883374,
+      "grad_norm": 5.288684844970703,
+      "learning_rate": 4.897959183673469e-06,
+      "loss": 2.6138,
+      "step": 1720
+    },
+    {
+      "epoch": 0.858560794044665,
+      "grad_norm": 5.238705635070801,
+      "learning_rate": 4.732487589630447e-06,
+      "loss": 2.5811,
+      "step": 1730
+    },
+    {
+      "epoch": 0.8635235732009926,
+      "grad_norm": 5.114046573638916,
+      "learning_rate": 4.567015995587424e-06,
+      "loss": 2.5933,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8684863523573201,
+      "grad_norm": 6.13997220993042,
+      "learning_rate": 4.401544401544402e-06,
+      "loss": 2.5869,
+      "step": 1750
+    },
+    {
+      "epoch": 0.8734491315136477,
+      "grad_norm": 5.258871555328369,
+      "learning_rate": 4.236072807501379e-06,
+      "loss": 2.5464,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8784119106699751,
+      "grad_norm": 5.823874473571777,
+      "learning_rate": 4.070601213458356e-06,
+      "loss": 2.6842,
+      "step": 1770
+    },
+    {
+      "epoch": 0.8833746898263027,
+      "grad_norm": 5.351442337036133,
+      "learning_rate": 3.905129619415334e-06,
+      "loss": 2.6312,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8883374689826302,
+      "grad_norm": 5.9153947830200195,
+      "learning_rate": 3.739658025372311e-06,
+      "loss": 2.5621,
+      "step": 1790
+    },
+    {
+      "epoch": 0.8933002481389578,
+      "grad_norm": 5.032203197479248,
+      "learning_rate": 3.5741864313292883e-06,
+      "loss": 2.6103,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8933002481389578,
+      "eval_loss": 2.285778284072876,
+      "eval_runtime": 108.9864,
+      "eval_samples_per_second": 155.469,
+      "eval_steps_per_second": 4.863,
+      "step": 1800
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2015,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 5,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 64,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68854b8e47ac69eebd49c5870121ff0ff8db3121fee3cb68ea673141818acc69
+size 5560