Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup
Paper • 2101.06983 • Published • 3
How to use finvector/sbert_raw_finmodern_tri_merged with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("finvector/sbert_raw_finmodern_tri_merged")
sentences = [
"In 2004, the company reported net sales of $5.34 billion and net income of $279.7 million, driven by core unit volume growth and favorable foreign currency translation of approximately $207 million. For 2005, the company anticipates core unit volume growth of 4% to 7%, expects interest expense between $50 million and $60 million, and projects an after-tax stock option expense of $8 million to $10 million for the second half of the year.",
"tter were $8.8 million and $16.1 million in 2017 and 2016, respectively. Home office lease termination/move - During 2017, we relocated our corporate office headquarters to a nearby suburb of Chicago. Move related after-tax charges were $7.9 million in 2017. These charges are presented in the corporate segment. Impact of U.S. tax reform - Consists of the tax expense from (a) adjusting December 31, 2017 initial estimates from the U.S. tax legislation passed in the fourth quarter of 2017 and (b) the on-going impact of such legislation - principally the partial taxation of foreign earnings, nondeductible executive compensation and entertainment expenses. Under the SEC Staff Accounting Bulletin No. 118 guidance, in our December 31, 2017 consolidated financial statements, we recognized provisional amounts for deferred income taxes and repatriation tax based on reasonable estimates and interpretations of the new tax legislation. The ultimate impact of the new tax legislation did differ from our estimated amounts as of December 31, 2017 amounts, due to, among other things, changes in interpretations and assumptions we made, or additional regulatory or accounting guidance that was issued with respect to the new tax legislation. In fourth quarter 2018, the IRS issued clarifying guidance related to the new tax legislation which resulted in us recognizing a tax benefit of $8.9 million in the quarter. Any additional taxes associated with the ongoing impact of the tax legislation had a de minimis impact on our cash taxes paid due to tax credits generated from our clean energy investments. Corporate legal entity restructuring - Consists of the tax benefit related to the release of valuation allowances that resulted from moving a legal entity within our subsidiary structure. Clean energy investments - We have investments in limited liability companies that own 29 clean coal production plants developed by us and five clean coal production plants we purchased from a third party on S",
"e a fine on us when its investigation is completed. We are unable to predict the effect of these matters at this time, although the effect could well be adverse and material. These matters are reported in Note 8 Contingencies, to the Consolidated Financial Statements on page 63 in our 2004 Annual Report to Shareholders. Outlook In 2005, we anticipate core unit volume growth in the range of 4 percent to 7 percent, driven in part by strong growth in the emerging markets of Asia, Latin America and Eastern Europe, subject to changes in global economic and market conditions. We expect a positive impact from price and mix, as well as a benefit from foreign currency translation. We anticipate that our selling price increases and ongoing productivity initiatives will offset rising raw material costs during the year. We expect to continue improving profitability as a result of our pricing actions, the successful consolidation of our roll materials business in Europe during 2004, and anticipated cost savings from our ongoing productivity initiatives, including Six Sigma. Offsetting these improvements to profitability during 2005, we expect to incur higher costs associated with our growth initiatives, including RFID. We expect increases in annual pension and medical costs to be in the range of $17 million to $18 million before taxes for 2005, due in part to an estimated increase of $13 million for pension expense. The projected increase in pension expense reflects the adjustment of our actuarial assumptions for both the U.S. and international plans. For example, our long-term rate of return on plan assets and discount rate were lowered by 25 basis points for the U.S. pension plans to 8.75 percent and 6.0 percent, respectively. Our estimate of pension expense will be impacted by changes in foreign currency translation. We estimate that interest expense will be between $50 million to $60 million for 2005, assuming expected interest rate increases, offset by projected reduction",
"ansportation to producers, brokers and other gas companies that own the natural gas transported, but are not the ultimate consumers. MichCons revenues and net income are impacted by weather and are concentrated in the first and fourth quarters of the year due to heating-related demand. MichCons operations are not dependent upon a limited number of customers, and the loss of any one or a few customers is not reasonably likely to have a material adverse effect on MichCon. We obtain our natural gas supply from various sources in different geographic areas under agreements that vary in both pricing and terms. 2004 Gas Distribution Contracted Supply Fixed Price Purchases 33 % Market Price Purchases 67 % 100 % Supply under contract represents approximately 75% of the expected 180 Bcf of supply requirements in 2004. We expect to meet the balance of gas supply requirements through open market purchases. As a result of varying demand primarily due to weather, MichCon may use existing gas in inventory to meet unanticipated customer obligations. Given the geographic diversity of supply, coupled with its 124 Bcf of storage capacity, MichCon believes it can meet the supply requirements for customers. MichCon has long-term firm transportation agreements expiring on various dates through 2011 for delivery of purchased natural gas to our distribution system. Regulation MichCon is subject to the regulatory jurisdiction of the MPSC, which issues orders pertaining to rates, recovery of certain costs, including the costs of regulatory assets, conditions of service, accounting and operating-related matters. MichCon is subject to the requirements of other regulatory agencies with respect to safety, the environment and health. In the late 1990s, the MPSC began an initiative designed to give all of Michigans natural gas customers added choices and the opportunity to benefit from lower gas costs resulting from competition. In 1999, the MPSC approved a comprehensive experimental three-yea"
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]This is a sentence-transformers model finetuned from sentence-transformers/bert-base-nli-mean-tokens on the finmodernbert-triplets dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
(1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
"As of December 31, 2009, Allstate Financial's total investments increased to $99.83 billion from $96.00 billion in 2008, driven by higher valuations in fixed income and equity securities despite a $5.83 billion net reduction in contractholder obligations. The company reduced municipal bond exposure by $1.88 billion and decreased portfolio duration by approximately 5% while maintaining a fixed income portfolio that was 93.7% investment grade.",
]
documents = [
" grade assets primarily corporate fixed income securities and bank loans, as a result of improved market conditions. \x95 Reduced our municipal bond exposure by 8.0% or $1.88 billion of amortized cost primarily through targeted dispositions, refunding and scheduled maturities. \x95 Managed our exposure to interest rate risk through macro hedging of tail risk and reductions in duration. Duration of the investment portfolio, including the effects of derivatives, declined by approximately 5% (.21 years) over the twelve month period. The decline in duration was the result of proceeds from asset 80 dispositions being reinvested in shorter duration assets and through the use of interest rate derivatives. Our investment portfolio duration is managed in relation to the liabilities of our business segments. \x95 Maintained risk mitigation programs that protected our portfolios through macro hedges against interest rate and equity market risk. The appreciation in fair values for fixed income and equity securities significantly exceeded the realized losses of $201 million from derivatives used for macro hedging programs. Investments outlook For 2010, the economies of the U.S. and other developed countries continue to face significant resistance with expectations for limited domestic growth, but with better prospects for growth in developing or emerging economies and markets. Financial markets are anticipated to see moderate returns for fixed income and equities with periods of higher volatility throughout the year, and moderate widening of credit spreads. We also expect continued weakness in commercial real estate fundamentals. In 2010, our return optimization and risk mitigation efforts will focus on the following areas: \x95 Optimizing our allocation of assets to reflect changes in Allstate Financial's liabilities. \x95 Continuing to explore global investments in areas of emerging opportunity with higher prospects for growth. \x95 Balancing our ongoing efforts to mitigate interest rate and eq",
'erest at a London Interbank Offered Rate (LIBOR) rate that is determined by the duration of such tranche plus an interest rate margin. The LIBOR variable component of the interest rate for each tranche is reset as such tranche matures and a new tranche is established. At December 31, 2016, the overall weighted average interest rate for Term Loan A was determined based upon the LIBOR interest rates in effect for all of the individual tranches plus the interest rate margin of 1.75%. At December 31, 2016, Term Loan B bears interest at LIBOR (floor of 0.75%) plus a margin of 2.75%. The Company is subject to LIBOR-based interest rate volatility on Term Loan B as the LIBOR-based component of the interest rate exceeded the floor of 0.75% as of December 31, 2016. The overall weighted average interest rate for Term Loan B was determined based upon the LIBOR interest rates in effect for all individual tranches plus the interest rate margin. The Company has several interest rate cap agreements that have the economic effect of capping the LIBOR variable component of the Company’s interest rate at a maximum of 3.50% on $3,500,000 of outstanding principal debt. The remaining $775,000 outstanding principal balance of Term Loan A would still be subject to LIBOR-based interest rate volatility. In addition, the Company maintains several forward interest rate cap agreements with notional amounts totaling $3,500,000, which will be effective June 29, 2018. The cap agreements will have the economic effect of capping the LIBOR variable component of the Company’s interest rate at a maximum of 3.50% on an equivalent amount of the Company’s debt. See below for further details. The Company is restricted from paying dividends under the terms of its senior secured credit facilities. During the year ended December 31, 2016, the Company made mandatory principal payments under its then existing senior secured credit facilities totaling $62,500 on Term Loan A and $35,000 on Term Loan B. Revolving l',
'nsecured short-term borrowings up to an aggregate of $200 million. As of December 31, 2001, commercial paper with a face amount of $100 million F-11 AMGEN INC. NOTES TO CONSOLIDATED FINANCIAL STATEMENTS--(Continued) was outstanding. These borrowings had maturities of less than one month and had effective interest rates averaging 1.9%. Commercial paper with a face amount of $100 million and with effective interest rates averaging 6.7% was outstanding at December 31, 2000. The Company has established a $500 million debt shelf registration statement. In December 1997, pursuant to this registration statement, the Company issued $100 million of debt securities that bear interest at a fixed rate of 6.5% and mature in 2007 (the "Notes") and established a $400 million medium-term note program. The Company may offer and issue medium-term notes from time to time with terms to be determined by market conditions. The Company had $100 million of debt securities outstanding at December 31, 2001 and 2000 that bear interest at a fixed rate of 8.1% and mature in 2097 (the "Century Notes"). These securities may be redeemed in whole or in part at the Company\'s option at any time for a redemption price equal to the greater of the principal amount to be redeemed or the sum of the present values of the principal and remaining interest payments discounted at a determined rate plus, in each case, accrued interest. In addition to the Notes and the Century Notes, debt securities outstanding at December 31, 2001 and 2000 include $23 million of debt securities that bear interest at a fixed rate of 6.2% and mature in 2003. The terms of the debt securities require the Company to meet certain debt to tangible net asset ratios and place limitations on liens and sale/leaseback transactions and, except with respect to the Notes and the Century Notes, place limitations on subsidiary indebtedness. The Company has an unsecured committed credit facility (the "credit facility") with five participating ba',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.6823, 0.1928, 0.3072]])
triplet-evalTripletEvaluator| Metric | Value |
|---|---|
| cosine_accuracy | 0.9904 |
query, positive, and negative| query | positive | negative | |
|---|---|---|---|
| type | string | string | string |
| modality | text | text | text |
| details |
|
|
|
| query | positive | negative |
|---|---|---|
Bank of America is subject to extensive oversight by U.S. and international regulators, including the SEC, CFTC, and the Federal Reserve, which impose capital, liquidity, and transaction limits on its banking and non-bank subsidiaries. The Corporation's ability to pay dividends and repurchase stock is contingent on maintaining regulatory capital levels, with its stress capital buffer increasing to 3.4% and its G-SIB surcharge expected to rise to 3.0% on January 1, 2024. |
ion by the SEC, Financial Industry Regulatory Authority and New York Stock Exchange, among others; our futures commission merchant subsidiaries supporting commodities and derivatives businesses in the U.S. are subject to regulation by and supervision of the U.S. Commodity Futures Trading Commission (CFTC), National Futures Association, the Chicago Mercantile Exchange, and in the case of the Banks, certain banking regulators; our insurance activities are subject to licensing and regulation by state insurance regulatory agencies; and our consumer financial products and services are regulated by the Consumer Financial Protection Bureau (CFPB). Our non-U.S. businesses are also subject to extensive regulation by various non-U.S. regulators, including governments, securities exchanges, prudential regulators, central banks and other regulatory bodies, in the jurisdictions in which those businesses operate. For example, our financial services entities in the United Kingdom (U.K.), Ireland and ... |
e corrections, increasing consumer and corporate leverage, increases in corporate bond spreads, rising or elevated unemployment levels, rising or elevated inflation, fluctuations in foreign exchange or interest rates, as well as widespread health emergencies or pandemics, extreme weather events and the impacts of climate change and domestic and global efforts to transition to a low-carbon economy. Significant economic or market stresses and disruptions typically have a negative impact on the business environment and financial markets, which could impact the underlying credit quality of our borrowers, counterparties and assets. Property value declines or asset price corrections could increase the risk of borrowers or counterparties defaulting or becoming delinquent in their obligations to us, and could decrease the value of the collateral we hold, which could increase credit losses. Credit risk could also be magnified by lending to leveraged borrowers or declining asset prices, includin... |
As of December 31, 2017, Ares Management, L.P. reported 7,369,430 exercisable options and 156,153 unvested phantom units with a fair value of $20.00 per unit, expecting to recognize $2.1 million in unrecognized compensation expense over 1.33 years. The Company also reported 12,400,000 units of Series A Preferred Equity outstanding, which are redeemable at $25.00 per unit starting June 30, 2021. |
— — — Exercised (54,500 ) 19.00 — 205 Expired (523,440 ) 19.00 — — Forfeited (1,159,169 ) 19.00 — — December 31, 2017 20,495,025 $ 18.99 6.09 $ 20,611 Exercisable at December 31, 2017 7,369,430 $ 19.00 5.62 $ 7,369 Aggregate intrinsic value represents the value of the Company’s closing unit price on the last trading day of the period in excess of the weighted-average exercise price multiplied by the number of options exercisable or expected to vest. The fair value of an award is affected by the Company’s unit price on the date of grant as well as other assumptions including the estimated volatility of the Company’s unit price over the term of the awards and the estimated period of time that F-53 Table of Contents Ares Management, L.P. Notes to the Consolidated Financial Statements (Dollars in Thousands, Except Unit Data and As Otherwise Noted) management expects employees to hold their unit options. The estimated period of time that management expects employees to hold their options w... |
es 2.1 % 2.4 % 2.3 % Amortization as a % of sales 5.4 % 5.6 % 4.0 % Revenue Performance 2017 vs. 2016 2016 vs. 2015 Total sales growth (GAAP) 6.5 % 62.0 % Less the impact of: Acquisitions and other (2.0 )% (59.0 )% Currency exchange rates (0.5 )% 0.5 % Core revenue growth (non-GAAP) 4.0 % 3.5 % 2017 Compared to 2016 During the first quarter of 2017, a product line was transferred from the Life Sciences segment to the Environmental & Applied Solutions segment. While this change is not material to segment results in total, the resulting change in sales growth has been included in the “Acquisitions and other” line in the table above. 34 Table of Contents Price increases in the segment contributed 0.5% to revenue growth on a year-over-year basis during 2017 as compared with 2016 and are reflected as a component of the change in core revenue growth. Core sales of the business’ broad range of mass spectrometers continued to grow on a year-over-year basis led by strong sales growth in the pha... |
The company operates across multiple jurisdictions, including the EU, UK, and Singapore, and is subject to various regulations such as MiFID II, UCITS V, and MAS requirements. Key entities include Threadneedle Singapore, Columbia Management, and Ameriprise Certificate Company, which are subject to specific oversight from the SEC, Minnesota Department of Commerce, and various state insurance regulators. |
luding as a regulated entity or as a service provider to, or a business receiving services from or engaging in transactions with, regulated entities. For example, within the EU and the UK we have been, or will be, addressing Solvency II, Markets in Financial Instruments Directive II (“MiFID II”), Market Abuse Regulation (“MAR”), Transparency Directive II (“TD II”), UCITS V, EMIR, Packaged Retail and Insurance-based Investment Products Regulation (“PRIIPs”), Fourth Money Laundering Directive (“MLD4”), Benchmarks Regulation, Money Market Fund Regulation (“MMFR”), Insurance Distribution Directive (“IDD”), Shareholder Rights Directive (“SRD”), General Data Protection Regulation (“GDPR”), UK Workplace Pensions Transaction Cost Disclosure, the Local Government Pension Scheme Advisory Board Transparency Code, the FCA’s Asset Management Market Study, and Senior Managers and Certification Regime (“SMCR”)and a financial transaction tax. On June 12, 2014, MiFID II was published in the Official Jo... |
where we conduct business, we are subject to extensive and complex governmental regulations which affect most aspects of our business, such as regulations governing the generation and distribution of electricity and environmental regulations. These regulations affect the operation, development, growth and ownership of our businesses. Regulations differ on a country by country basis and are based upon the type of business we operate in a particular country. Regulation of our Generation Businesses Our Generation businesses operate in two different types of regulatory environments: Market Environments. In market environments, sales of electricity may be made directly on the spot market, under negotiated bilateral contracts, or pursuant to PPAs. The spot markets are typically administered by a central dispatch or system operator who seeks to optimize the use of the generation resources throughout an interconnected system (cost of the least expensive next generation plant required to meet s... |
CachedMultipleNegativesRankingLoss with these parameters:{
"scale": 20.0,
"similarity_fct": "cos_sim",
"mini_batch_size": 4,
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
query, positive, and negative| query | positive | negative | |
|---|---|---|---|
| type | string | string | string |
| modality | text | text | text |
| details |
|
|
|
| query | positive | negative |
|---|---|---|
The passage provides a detailed property listing for various real estate holdings across San Diego, Seattle, Maryland, and the Research Triangle, detailing rentable square footage (RSF), occupancy percentages, and annual rental revenue for specific 'Megacampus' developments and individual properties. Key highlights include the San Diego portfolio with 8,303,960 total RSF at 96.3% occupancy, the Seattle portfolio with 3,414,389 total RSF at 92.4% occupancy, the Maryland portfolio with 3,849,928 total RSF at 95.7% occupancy, and the Research Triangle portfolio with 3,802,204 total RSF at 97.4% occupancy. |
,463 426,927 — 2,021,390 10 86,469 98.0 98.0 9880 (2) , 10210, 10260, 10290, and 10300 Campus Point Drive and 4135, 4155, 4161, 4224, and 4242 Campus Point Court Megacampus: 5200 Illumina Way (1) 792,687 — — 792,687 6 29,978 100.0 100.0 9625 Towne Centre Drive (1) 163,648 — — 163,648 1 6,520 100.0 100.0 University Town Center 2,550,798 426,927 — 2,977,725 17 122,967 98.8 98.8 Sorrento Mesa Megacampus: SD Tech by Alexandria (1) 878,805 253,079 — 1,131,884 12 39,988 93.6 93.6 9605, 9645, 9675, 9725, 9735, 9808, 9855, and 9868 Scranton Road, 5505 Morehouse Drive (2) , and 10055, 10065, and 10075 Barnes Canyon Road Megacampus: Sequence District by Alexandria 801,575 — — 801,575 7 28,766 100.0 100.0 6260, 6290, 6310, 6340, 6350, 6420, and 6450 Sequence Drive Pacific Technology Park (1) 544,352 — — 544,352 5 9,352 92.8 92.8 9389, 9393, 9401, 9455, and 9477 Waples Street Summers Ridge Science Park (1) 316,531 — — 316,531 4 11,521 100.0 100.0 9965, 9975, 9985, and 9995 Summers Ridge Road Scrip... |
8,100 35,493 (4,759 ) 30,734 2007 2007 Alexandria Center™ for Life Science Greater New York City — — — 516,879 — 516,879 516,879 (32,334 ) 484,545 2010-2013 2006 100 Phillips Parkway Greater New York City 9,640 (5) 1,840 2,298 14,807 1,840 17,105 18,945 (7,534 ) 11,411 1999 1998 102 Witmer Road Greater New York City — 1,625 19,715 5,641 1,625 25,356 26,981 (5,434 ) 21,547 2002 2006 701 Veterans Circle Greater New York City — 1,468 7,885 24 1,468 7,909 9,377 (1,277 ) 8,100 2007 2007 5100 Campus Drive Greater New York City — 327 2,117 1,189 327 3,306 3,633 (1,169 ) 2,464 1989 1998 9800 Medical Center Drive Maryland 76,000 5,523 99,696 93,752 5,523 193,448 198,971 (35,579 ) 163,392 2010-2013 2004 1330 Piccard Drive Maryland — 2,800 11,533 28,530 2,800 40,063 42,863 (11,373 ) 31,490 2005 1997 1500/1550 East Gude Drive Maryland — 1,523 7,731 4,802 1,523 12,533 14,056 (4,597 ) 9,459 2003/1995 1997 14920/15010 Broschart Road Maryland 6,006 (11) 4,904 15,846 3,778 4,904 19,624 24,528 (2,648 ) ... |
The company outlines global regulatory requirements, including the EU's PRIIPs Key Information Document (KID) and potential Financial Transaction Tax (FTT), and details specific oversight by authorities in Canada (OSFI, MASCA), Singapore (MAS), Australia (ASIC), Hong Kong (SFC), India (SEBI, RBI), and Japan (JFSA). Non-compliance with these diverse regional laws and standards may result in administrative penalties, fines, or the suspension and revocation of operating licenses. |
to be implemented through the strengthening of MiFID standards for non-insurance PRIIPs, revisions to the Insurance Mediation Directive’s selling standard for all insurance-based PRIIPs and new investor disclosure requirements for all PRIIPs through the PRIIPs Key Information Document (“KID”) regulation, which was approved by the European Parliament in April 2014. It requires manufacturers of these products to draw up a KID which can be no longer than three pages in length and must be written in simple language. The regulation allows UCITS providers, who are already required to produce the UCITS Key Investor Information Document, a transitional period of five years from enactment during which they will be exempt from its terms. In addition, the European Commission ’ s proposal for a financial transaction tax (“FTT”) in the EU, which would, if approved by all EU member states, apply to all financial transactions where at least one party is established in an EU member state and either t... |
ovember 2007 authorization, the Board of Directors has authorized the repurchase of up to $100 million of our common stock. The new authorization is effective from December 1, 2008 to November 30, 2009. During fiscal 2008, the Board of Directors approved two quarterly cash dividends of $0.15 per common share followed by two quarterly cash dividends of $0.075 per common share, the last of which was paid on August 28, 2008 to stockholders of record on August 18, 2008. Cash dividends of $0.15 per common share were paid in each quarter of fiscal 2007. On November 20, 2008, the Board of Directors approved a cash dividend of $0.0375 per common share, payable on December 18, 2008, to stockholders of record on December 8, 2008. NOTE J EMPLOYEE BENEFIT PLANS Deferred Compensation The Company has a 401(k) plan for all Company employees who have been with the Company for a period of six months or more. The Company matches portions of employees voluntary contributions. Additional employer contr... |
Abbott's Pharmaceutical Products segment markets prescription drugs to physicians, managed care providers, and government agencies, facing competition from technological innovation, price, and generic drug substitution. The Diagnostic Products segment offers systems and tests for hospitals and laboratories, including immunoassay and chemistry systems, while the Ross Products segment provides pediatric and adult nutritionals like Similac and Ensure, as well as specialty pharmaceuticals like Survanta. |
ublic warehouses. This segment directs its primary marketing efforts toward securing the prescription of Abbott's brand of products by physicians. Managed care providers (for example, health maintenance organizations and pharmacy benefit managers) and state and federal governments and agencies (for example, the Department of Veterans Affairs and the Department of Defense) are also important customers. Competition in the Pharmaceutical Products segment is generally from other health care and pharmaceutical companies. The search for technological innovations in pharmaceutical products is a significant aspect of competition in this segment. The introduction of new products by competitors and changes in medical practices and procedures can result in product obsolescence in the Pharmaceutical Products segment. Price can also be a factor. In addition, the substitution of generic drugs for the brand prescribed has increased competitive pressures on pharmaceutical products which are off-patent... |
erformed by third-party contract manufacturers. We also conduct certain finish activities in the Netherlands. (See Item 2. Properties.) Clinical Manufacturing Clinical bulk, formulation, fill and finish manufacturing facilities are operated primarily in our Thousand Oaks, California location. (See Item 2. Properties.) Certain finishing activities for our clinical products are performed in the Netherlands. In addition, we also utilize third-party contract manufacturers for certain of our clinical products. See Item 1A. Risk Factors We perform a substantial amount of our commercial manufacturing activities at our Puerto Rico manufacturing facility and a substantial amount of our clinical manufacturing activities at our Thousand Oaks, California manufacturing facility; if significant natural disasters or production failures occur at the Puerto Rico facility, we may not be able to supply these products or, at the Thousand Oaks facility, we may not be able to continue our clinical trials.... |
CachedMultipleNegativesRankingLoss with these parameters:{
"scale": 20.0,
"similarity_fct": "cos_sim",
"mini_batch_size": 4,
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
per_device_train_batch_size: 16num_train_epochs: 10learning_rate: 2e-05warmup_steps: 0.1bf16: Trueper_device_eval_batch_size: 16load_best_model_at_end: Trueseed: 777data_seed: 777prompts: {'query': '', 'positive': '', 'negative': ''}batch_sampler: no_duplicatesper_device_train_batch_size: 16num_train_epochs: 10max_steps: -1learning_rate: 2e-05lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0.1optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Truefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 16prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Trueignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 777data_seed: 777use_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Nonedataloader_multiprocessing_context: Nonedataloader_in_order: Trueremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonelocal_rank: -1prompts: {'query': '', 'positive': '', 'negative': ''}batch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}warmup_ratio: None| Epoch | Step | Training Loss | Validation Loss | triplet-eval_cosine_accuracy |
|---|---|---|---|---|
| -1 | -1 | - | - | 0.9050 |
| 1.0 | 731 | 1.1845 | 0.4494 | 0.9840 |
| 2.0 | 1462 | 0.3394 | 0.2883 | 0.9880 |
| 3.0 | 2193 | 0.2444 | 0.2543 | 0.9888 |
| 4.0 | 2924 | 0.2052 | 0.2394 | 0.9900 |
| 5.0 | 3655 | 0.1919 | 0.2333 | 0.9900 |
| 6.0 | 4386 | 0.1774 | 0.2302 | 0.9904 |
| 7.0 | 5117 | 0.1755 | 0.2292 | 0.9896 |
| 8.0 | 5848 | 0.1785 | 0.2285 | 0.9904 |
| 9.0 | 6579 | 0.1773 | 0.2284 | 0.9904 |
| 10.0 | 7310 | 0.1809 | 0.2284 | 0.9904 |
| -1 | -1 | - | - | 0.9904 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{gao2021scaling,
title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
year={2021},
eprint={2101.06983},
archivePrefix={arXiv},
primaryClass={cs.LG}
}