Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Paper • 1908.10084 • Published • 14
How to use shreyaspullehf/superconductor-search-v2 with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("shreyaspullehf/superconductor-search-v2")
sentences = [
"magnetic order superconductor",
"arXiv:0802.4093v2 [cond-mat.str-el] 27 Feb 2009 Meissner effect without superconductivity from a chiral d-density wave P. Kotetes∗and G. Varelogiannis† Department of Physics, National Technical University of Athens, GR-15780 Athens, Greece We demonstrate that the formation of a chiral d-density wave (CDDW) state generates a Topolo- gical Meissner effect (TME) in the absence of any kind of superconductivity. The TME is identical to the usual superconducting Meissner effect but it appears only for magnetic fields perpendicular to the plane while it is absent for in plane fields. The observed enhanced diamagnetic signals in the non-superconducting pseudogap regime of the cuprates may find an alternative interpretation in terms of a TME, originating from a chiral d-density wave pseudogap. PACS numbers: 75.20.-g, 71.27.+a, 74.72.-h The Meissner effect is considered to be the most direct signature of superconductivity [1]. However, the surpris- ing observations of such enhanced diamagnetic signals [2] well above the superconducting transition tempera- ture in the pseudogap regime of the cuprates [3], con- stitute a fascinating puzzle. There are two proposals for the nature of this regime that appear to dominate. The first, associates the pseudogap with a dx2−y2 density wave (DDW) [4, 5], also called orbital antiferromagnet [6, 7], which normally competes with superconductivity. The second associates the pseudogap with spontaneous vortex-antivortex unbinding leading to incoherent super- conductivity [8] that should persist well above the su- perconducting Tc. This theory is reminiscent of the well known Kosterlitz-Thouless transition [9]. The available ARPES [10] and STM [11] experiments cannot differentiate a SC from a density wave (DW) gap, and therefore appear somehow incapable in settling di- rectly the issue. On the other hand, the unusual Nernst effect and most importantly, the enhanced diamagnetic signal that accompanies it for a very large temperature region above the SC critical temperature [2], has been considered as a major argument in favor of the incoher- ent SC scenario. In fact, the enhanced diamagnetism is viewed as a signature of the usual Meissner effect associ- ated solely with the SC state, and would therefore contra- dict the dx2−y2 density wave scenario since no Meissner effect was expected in that case [7]. In this letter we put forward the Topological Meissner effect (TME), that results from a chiral dxy + idx2−y2 density wave (CDDW) state. In fact, the Nernst region of the pseudogap regime may well be associated with a CDDW. The most intriguing property of a CDDW is that parity (P) and time-reversal (T ) violation induces Chern-Simons terms in the effective action of the elec- tromagnetic field, providing the possibility of the TME and the Spontaneous Quantum Hall effect (SQHE) ear- lier discussed [12, 13, 14, 15, 16, 17, 18]. As we shall demonstrate, the TME is described by the same equa- tion we find in the usual Meissner effect of a supercon- ∗Electronic address: pkotetes@central.ntua.gr †Electronic address: varelogi@central.ntua.gr ductor. Though, its origin is radically different. In our system we encounter the realization of Parity Anomaly [13, 19], with the emerging Chern-Simons terms provi- ding a topological mass to the electromagnetic field, in a gauge invariant manner [20, 21]. Moreover, the pos- session of chirality perpendicular to the plane, implies that the TME is strongly anisotropic. Particularly, it takes place for magnetic fields perpendicular to the plane while it is absent for in plane fields, in accordance with the experimental observations [2]. Note finally that a chi- ral d-density wave state has also been shown recently [22] to explain the experimental results concerning the Polar Kerr effect in YBCO [23]. In order to demonstrate how the TME arises, we shall consider the following BCS hamiltonian for the CDDW HCDDW = 1 2 X k \u0010 ∆kc† kck+Q + ∆∗ kc† k+Qck \u0011 , (1) which describes a dxy+idx2−y2 state characterized by the wave-vector Q = (π, π), which is commensurate to the lattice (k + 2Q = k). Since spin degrees of freedom do not get involved we have considered spinless electrons, so that all our results will refer to one spin component. Fur- thermore, we use gµν = (1, −1, −1), ki = k = (kx, ky), kµ = k = (ω, k), qµ = q = (q0, q), µ = 0, 1, 2, i = 1, 2, e > 0, ℏ= 1 and we assume that repeated indices are summed. In the derivation of the Chern-Simons terms we shall restrict ourselves to the zero temperature case while necessary extensions to finite temperatures will be afterwards performed. In addition, the summation in k−space is all over the whole 1st Brillouin zone rather than the reduced Brillouin zone. This implies that the operators ck and ck+Q do not describe independent de- grees of freedom. In Eq.(1) we have introduced the CDDW order param- eter ∆k = η∆sin kx sin ky+i∆ \u0000cos kx −cos ky \u0001 , where ∆ is the modulus of the idx2−y2 order parameter, η defines the relative magnitude of the two components and also determines the direction of the chirality of the state. The chiral character of the state implies the existence of an intrinsic angular momentum in k−space, perpendicular to the plane, originating from P −T violation. Specifi- cally, the dx2−y2 component violates T as it is imaginary, 2 while the dxy component is odd under P in two dimen- sions, which is defined as (kx, ky) →(kx, −ky). In order to obtain the total electronic Hamiltonian H, we have to add the corresponding kinetic part Hkin. For the kinetic part we keep only the nearest neighbors hop- ping term ǫk = −t \u0000cos kx + cos ky \u0001 satisfying the nest- ing condition ǫk+Q = −ǫk, while we also set the chem- ical potential equal to zero. Our approximation can be justified by considering that our system is close to half- filling. Under these conditions the excitation spectrum consists of two bands which are fully gapped leading to the topological quantization of the Hall conductance [12, 13, 14, 15, 16, 17, 18], which is the coefficient of the Chern-Simons terms. Omitting the next nearest neigh- bors hopping term δk = t′ cos kx cos ky does not alter qualitatively the occurrence of the TME. However, its inclusion would destroy the quantization of the Hall con- ductance, as in this case, the system is not fully gapped. Similar effects would arise in the presence of disorder or by including the z-axis hopping term. Under this conditions, the total Hamiltonian of the system becomes H = 1 2 P k h ǫk \u0010 c† kck −c† k+Qck+Q \u0011 + \u0010 ∆kc† kck+Q + h.c. \u0011i . We obtain a compact representa- tion of H by introducing the spinor Ψ† k = 1 √ 2(c† k c† k+Q), the isospin Pauli matrices τ and the vector gk ≡ (Re∆k, −Im∆k, ǫk). This yields H = P k Ψ† k gk · τ Ψk. The latter indicates that the ground state of the sys- tem depends on the orientation of the g vector in isospin space. As a result, this hamiltonian supports skyrmion solutions which imply the presence of a Chern-Simons action (see e.g. [18]). To reveal the emerging Chern-Simons terms, we have to take into account the fluctuations of the U(1) gauge field Aµ. We add to the Hamiltonian the term Hem = R d2q (2π)2 P k Ψ† k+qΓµ k+q,kAµ(q)Ψk − R d2q (2π)2 P k Ψ† k+q e2 2mAi(−q)Ai(q)Ψk, which describes the interaction of the gauge field with the electrons. We have introduced the paramagnetic interaction vertex Γµ k+q,k = −(e , e ∂ ∂ki gk · τ), where µ = 0, 1, 2 and i = 1, 2. At one-loop level, the effective action Sem is given by the relation Sem = 1 2 R d3q (2π)3 Aµ(−q)Πµν(q)Aν(q), with the Polarization tensor Πµν, defined as Πµν(q) = i 2 R k T r \u0010 GkΓµ k,k+qGk+qΓν k+q,k \u0011 −e2 m ρeδi,j. ρe is the two- dimensional electronic density (without including spin), T r denotes trace over isospin indices, Gk is the CDDW fermionic propagator and we have used the abbreviation R k = R dω 2π P k. Computing Πµν up to linear order in q, yields the Chern-Simons action SCS = Z d3x σxy 4 εµνλAµF νλ, (2) with Fµν = ∂µAν −∂νAµ. The coefficient of the Chern- Simons action is the Hall conductance σxy. It can be shown that it is a topological invariant, reflecting the existence of a topologically non trivial, P −T violating ground state (see e.g. [18]). Using Eq.(2) we obtain σxy = i 2!ε0ji ∂Π0i ∂qj = e2 4π b N = e2 2π , (3) where we have introduced the winding number of the unit vector ˆgk = gk/| gk |, b N = 1 4π Z d2k ˆgk · ∂ˆgk ∂kx × ∂ˆgk ∂ky ! , (4) which is equal to 2, because the order parameter com- ponents are eigenfunctions of the angular momentum in k−space with eigenvalue l = 2. In the case of a perfect gap, the Hall conductance origi- nates only from the chirality b N of the lower energy band, E− k = −|gk|, which is fully occupied. In the same time, the upper band, E+ k = +|gk|, is totally empty while it is characterized by opposite chirality. Apparently, if both bands were equally occupied then σxy would be equal to zero. In the general case, the two bands, have different occupation numbers n−and n+, yielding a non-quantized Hall conductance σxy = e2 2π(n−−n+). Deviations from nesting, disorder or a chemical potential generally lead to such an effect. It is desirable to comprehend, even crudely, the effect of these parameters on the Hall con- ductance and the TME. For this purpose we consider that a finite chemical po- tential is added to the system. We shall consider that its magnitude is of the order of min|gk|. This minimum is realized at the points k0 = (± π 2 , ± π 2 ), when η << 1. In this case, we may linearize the spectrum about these points so to obtain an approximate analytical solution. The two energy bands are described by the dispersions E± k = −µ± p m2 + (v0 · δk)2, with m = min|gk| = |gk0|, v0 the velocity at these points and δk = k−k0. If |µ| ≥m and µ < 0, hole-pockets arise in the lower band decreas- ing the full occupancy from n−= 1 to n−= 1−nex, with nex the portion of the empty states. On the other hand, if µ ≥m, electron pockets emerge in the upper band ris- ing its occupancy from zero. However, if we take into consideration that the two bands have opposite chirality, it is evident that in both cases, the effect is the same. Consequently, σxy(µ) = σxy(1 −nex). The portion of the empty states will be determined by the area of the el- lipses defined by the four hole-pockets. Straightforward calculations yield the simple relation nex = (µ2 −m2)/2πt∆. (5) We observe that for small values of |µ|, compared to t and ∆, the effect of doping is negligible. We are now in position to obtain the equations of mo- tion of the gauge field which will allow us to discuss the TME in a Hall bar geometry setup. We consider that the 3 FIG. 1: (Color online) (a) Influence of doping on the Topo- logical Meissner effect. The relative change of magnetization hardly reaches 1% in the presence of a small chemical po- tential (∆= 20meV , t = 500meV ). (b) The magnetic field screening as a function of the position on the Hall bar extend- ing from −lx to +lx, for different values of the penetration depth λ over lx. The magnetic field is totally expelled from the sample when lx/λ >> 1 exactly as in the superconducting case. Hall bar has dimensions Lx = 2lx, Ly >> Lx extending from −lx to lx on the x-axis. The relation Ly >> Lx in- dicates that there is negligible y-dependence of the",
"QCage - A New Microwave-cavity Sample Holder for High-fidelity Qubit Measurements | In this video, you will learn about a new sample holder system, targeted at superconducting quantum processors with tens of qubits.\n\nThe QCage is a sample holder system for microwave resonator-based quantum devices that encloses the sample chip in an all-surrounding microwave cavity, with a total number of 24 coplanar transmission lines optimized for frequencies up to 18 GHz. The QCage is designed to hold the chip suspended inside the cavity, supported only by four corner pedestals and clamped d",
"Elastic neutron scattering simultaneously probes both the crystal structure\nand magnetic order in a material. Inelastic neutron scattering measures phonons\nand magnetic excitations. Here, we review the average composition, crystal\nstructure and magnetic order in the 245 family of Fe-based superconductors and\nin related insulating compounds from neutron diffraction works. A\nthree-dimensional phase-diagram summarizes various structural, magnetic and\nelectronic properties as a function of the sample composition. A high pressure\nphase diagram for the superconductor is also provided. Magnetic excitations and\nthe theoretic Heisenberg Hamiltonian are provided for the superconductor.\nIssues for future works are discussed."
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'superconductivity',
'We highlight the reproducibility and level of control over the electrical\nproperties of YBa$_2$Cu$_3$O$_7$ Josephson junctions fabricated with\nirradiation from a focused helium ion beam. Specifically, we show the results\nof electrical transport properties for several junctions fabricated using a\nlarge range of irradiation doses. At the lower end of this range, junctions\nexhibit superconductor-normal metal-superconductor (SNS) Josephson junction\nproperties. However, as dose increases there is a transition to electrical\ncharacteristics consistent with superconductor-insulator-superconductor (SIS)\njunctions. To investigate the uniformity of large numbers of helium ion\nJosephson junctions we fabricate arrays of both SNS and SIS Josephson junctions\ncontaining 20 connected in series. Electrical transport properties for these\narrays reveal very uniform junctions with no appreciable spread in critical\ncurrent or resistance.',
'Non-invasive magnetic field sensing using optically - detected magnetic\nresonance of nitrogen-vacancy (NV) centers in diamond was used to study spatial\ndistribution of the magnetic induction upon penetration and expulsion of weak\nmagnetic fields in several representative superconductors. Vector magnetic\nfields were measured on the surface of conventional, Pb and Nb, and\nunconventional, LuNi$_2$B$_2$C, Ba$_{0.6}$K$_{0.4}$Fe$_2$As$_2$,\nBa(Fe$_{0.93}$Co$_{0.07}$)$_2$As$_2$, and CaKFe$_4$As$_4$, superconductors,\nwith diffraction - limited spatial resolution using variable - temperature\nconfocal system. Magnetic induction profiles across the crystal edges were\nmeasured in zero-field-cooled (ZFC) and field-cooled (FC) conditions. While all\nsuperconductors show nearly perfect screening of magnetic fields applied after\ncooling to temperatures well below the superconducting transition, $T_c$, a\nrange of very different behaviors was observed for Meissner expulsion upon\ncooling in static magnetic field from above $T_c$. Substantial conventional\nMeissner expulsion is found in LuNi$_2$B$_2$C, paramagnetic Meissner effect\n(PME) is found in Nb, and virtually no expulsion is observed in iron-based\nsuperconductors. In all cases, good correlation with macroscopic measurements\nof total magnetic moment is found. Our measurements of the spatial distribution\nof magnetic induction provide insight into microscopic physics of the Meissner\neffect.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.3505, 0.3544],
# [0.3505, 1.0000, 0.6777],
# [0.3544, 0.6777, 1.0000]])
superconductor-evalInformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.7 |
| cosine_accuracy@3 | 0.8 |
| cosine_accuracy@5 | 0.9 |
| cosine_accuracy@10 | 1.0 |
| cosine_precision@1 | 0.7 |
| cosine_precision@3 | 0.3667 |
| cosine_precision@5 | 0.24 |
| cosine_precision@10 | 0.13 |
| cosine_recall@1 | 0.5 |
| cosine_recall@3 | 0.7 |
| cosine_recall@5 | 0.8 |
| cosine_recall@10 | 0.9 |
| cosine_ndcg@10 | 0.7651 |
| cosine_mrr@10 | 0.7861 |
| cosine_map@100 | 0.7098 |
sentence_0 and sentence_1| sentence_0 | sentence_1 | |
|---|---|---|
| type | string | string |
| details |
|
|
| sentence_0 | sentence_1 |
|---|---|
codimension two lump solutions in string field theory and ta |
We present some solutions for lumps in two dimensions in level-expanded |
superconductivity explained |
We review the current understanding of superconductivity in the |
erez berg |
Erez Berg- Theory of Strange Metals | Understanding "strange metal" phenomena - metallic behavior that deviates from that expected of an ordinary Fermi liquid down to the lowest measurable temperatures - is among the most puzzling open problems in condensed matter physics. Such phenomena are observed across many different strongly correlated materials. They seem tied to other interesting phenomena, such as quantum criticality and unconventional superconductivity. I will describe theoretical advances in understanding the possible or |
MultipleNegativesRankingLoss with these parameters:{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false
}
eval_strategy: stepsper_device_train_batch_size: 16per_device_eval_batch_size: 16num_train_epochs: 4multi_dataset_batch_sampler: round_robinoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: stepsprediction_loss_only: Trueper_device_train_batch_size: 16per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 5e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1num_train_epochs: 4max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.0warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falsebf16: Falsefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Falseignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedeepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthproject: huggingfacetrackio_space_id: trackioddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsehub_revision: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: noneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseliger_kernel_config: Noneeval_use_gather_object: Falseaverage_tokens_across_devices: Trueprompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: round_robinrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | superconductor-eval_cosine_ndcg@10 |
|---|---|---|
| 1.0 | 256 | 0.7651 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Base model
nreimers/MiniLM-L6-H384-uncased