Instructions to use hanzceo/SimpleEmbed-dev1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hanzceo/SimpleEmbed-dev1 with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("hanzceo/SimpleEmbed-dev1")

sentences = [
    "<p dir=\"auto\"><strong>Is your feature request related to a problem? Please describe.</strong><br>\nscipy.cluster.hierarchy.linkage uses double (float64) to store and do its computation for hierarchical clustering. However, I have a very large dataset (292000x292000) that I would like to perform hclust on but my computer is RAM limited. I have 252GB RAM and I think the clustering algorithm should be able to work on my dataset when all values are stored and computed using float16s instead.</p>\n<p dir=\"auto\">For large datasets on machines with insufficient RAM to store and compute on Arrays of float64s, it would be awesome if computation could be done on a different precision float to reduce the memory footprint.</p>\n<p dir=\"auto\">Additionally, adding choices for datatypes could be very useful for almost all scipy functions.</p>\n<p dir=\"auto\"><strong>Describe the solution you'd like</strong><br>\nAllow for an argument to specify what datatype you'd like to use (e.g. np.float64, np.float32, np.float16)</p>\n<p dir=\"auto\">The argument could be like dtype='np.double' by default but changable to whatever datatype is chosen.</p>",
    "<p dir=\"auto\">One representative error:</p>\n<div class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"torch/csrc/autograd/functions/init.cpp:220:37: error: address of overloaded function 'getTupleAttr' does not match required type '_object *(_object *, void *)'\n  {(char*)&quot;output_padding&quot;, (getter)getTupleAttr&lt;ConvBackwardBackward, std::vector&lt;int&gt;, ConvParams,\n                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\ntorch/csrc/autograd/functions/init.cpp:82:11: note: candidate template ignored: invalid explicitly-specified argument for template parameter 'Convert'\nPyObject* getTupleAttr(PyObject* obj, void* _unused)\"><pre class=\"notranslate\"><code class=\"notranslate\">torch/csrc/autograd/functions/init.cpp:220:37: error: address of overloaded function 'getTupleAttr' does not match required type '_object *(_object *, void *)'\n  {(char*)\"output_padding\", (getter)getTupleAttr&lt;ConvBackwardBackward, std::vector&lt;int&gt;, ConvParams,\n                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\ntorch/csrc/autograd/functions/init.cpp:82:11: note: candidate template ignored: invalid explicitly-specified argument for template parameter 'Convert'\nPyObject* getTupleAttr(PyObject* obj, void* _unused)\n</code></pre></div>\n<p dir=\"auto\">The cause of the problem is <a class=\"commit-link\" data-hovercard-type=\"commit\" data-hovercard-url=\"https://github.com/pytorch/pytorch/commit/aa911939a328eff55c9b28b39ed3c43507ba8a2a/hovercard\" href=\"https://github.com/pytorch/pytorch/commit/aa911939a328eff55c9b28b39ed3c43507ba8a2a\"><tt>aa91193</tt></a>:</p>\n<div class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"   {(char*)&quot;output_padding&quot;, (getter)getTupleAttr&lt;ConvForward, std::vector&lt;int&gt;, ConvParams,\n-                                         &amp;ConvParams::output_padding, long, PyInt_FromLong&gt;, NULL, NULL, NULL},\n+                                         &amp;ConvParams::output_padding, int64_t, PyInt_FromLong&gt;, NULL, NULL, NULL},\"><pre class=\"notranslate\"><code class=\"notranslate\">   {(char*)\"output_padding\", (getter)getTupleAttr&lt;ConvForward, std::vector&lt;int&gt;, ConvParams,\n-                                         &amp;ConvParams::output_padding, long, PyInt_FromLong&gt;, NULL, NULL, NULL},\n+                                         &amp;ConvParams::output_padding, int64_t, PyInt_FromLong&gt;, NULL, NULL, NULL},\n</code></pre></div>\n<p dir=\"auto\">It seems that on clang, changing the type parameter here is sufficient to cause template instantiation to fail.</p>\n<p dir=\"auto\">Maybe the easiest way to fix this is to write a more portable version of PyInt_FromLong (and friends) which always returns <code class=\"notranslate\">int64_t</code>.</p>",
    "<p dir=\"auto\">I try the scipy ward clustering, when calculating linkage, it report follow error:</p>\n<div class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"ward_h = linkage(X, method='ward', metric='euclidean')\nPython(2557,0x7fff732cc310) malloc: *** mach_vm_map(size=18446744067627675648) failed (error code=3)\n*** error: can't allocate region\n*** set a breakpoint in malloc_error_break to debug\n---------------------------------------------------------------------------\nMemoryError                               Traceback (most recent call last)\n&lt;ipython-input-10-769ae7c53f7c&gt; in &lt;module&gt;()\n----&gt; 1 ward_h = linkage(X, method='ward', metric='euclidean')\n\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.pyc in linkage(y, method, metric)\n    652             Z = np.zeros((n - 1, 4))\n    653             _hierarchy_wrap.linkage_euclid_wrap(dm, Z, X, m, n,\n--&gt; 654                                               int(_cpy_euclid_methods[method]))\n    655     return Z\n    656 \n\nMemoryError: out of memory while computing linkage\"><pre class=\"notranslate\"><code class=\"notranslate\">ward_h = linkage(X, method='ward', metric='euclidean')\nPython(2557,0x7fff732cc310) malloc: *** mach_vm_map(size=18446744067627675648) failed (error code=3)\n*** error: can't allocate region\n*** set a breakpoint in malloc_error_break to debug\n---------------------------------------------------------------------------\nMemoryError                               Traceback (most recent call last)\n&lt;ipython-input-10-769ae7c53f7c&gt; in &lt;module&gt;()\n----&gt; 1 ward_h = linkage(X, method='ward', metric='euclidean')\n\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.pyc in linkage(y, method, metric)\n    652             Z = np.zeros((n - 1, 4))\n    653             _hierarchy_wrap.linkage_euclid_wrap(dm, Z, X, m, n,\n--&gt; 654                                               int(_cpy_euclid_methods[method]))\n    655     return Z\n    656 \n\nMemoryError: out of memory while computing linkage\n</code></pre></div>\n<p dir=\"auto\">How can I solve this？</p>\n<p dir=\"auto\">The data set I use is here: <a href=\"https://dl.dropboxusercontent.com/u/68126956/df.csv\" rel=\"nofollow\">https://dl.dropboxusercontent.com/u/68126956/df.csv</a>.</p>\n<p dir=\"auto\">Thanks.</p>",
    "<p dir=\"auto\">Make sure these boxes are checked before submitting your issue - thank you!</p>\n<ul dir=\"auto\">\n<li>[yes ] I have checked the superset logs for python stacktraces and included it here as text if any</li>\n<li>[yes ] I have reproduced the issue with at least the latest released version of superset</li>\n<li>[yes ] I have checked the issue tracker for the same issue and I haven't found one similar</li>\n</ul>\n<h3 dir=\"auto\">Superset version</h3>\n<p dir=\"auto\">0.19.1</p>\n<h3 dir=\"auto\">Expected results</h3>\n<p dir=\"auto\">I try to draw mapbox in superset. I have dataset with column Latitude and Longitude and use it in respective field.</p>\n<h3 dir=\"auto\">Actual results</h3>\n<p dir=\"auto\">TypeError: &lt;superset.connectors.druid.models.DruidMetric object at 0xefbea90&gt; is not JSON serializable</p>\n<h3 dir=\"auto\">Steps to reproduce</h3>\n<p dir=\"auto\"><a target=\"_blank\" rel=\"noopener noreferrer nofollow\" href=\"https://user-images.githubusercontent.com/13684386/31226202-8d89286c-a9d5-11e7-8d8b-eb7e9d6c4d77.png\"><img src=\"https://user-images.githubusercontent.com/13684386/31226202-8d89286c-a9d5-11e7-8d8b-eb7e9d6c4d77.png\" alt=\"togithub\" style=\"max-width: 100%;\"></a></p>\n<p dir=\"auto\">Anyone  already have the same problem?<br>\nThanks</p>"
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Notebooks
Google Colab
Kaggle

SimpleEmbed-dev1 / README.md

hanzceo

Add new SentenceTransformer model

664401b verified 9 days ago

preview code

Raw

History Blame Contribute Delete

52 kB

metadata

language:
  - en
license: mit
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:219902
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      <p dir="auto"><strong>Is your feature request related to a problem? Please
      describe.</strong><br>

      scipy.cluster.hierarchy.linkage uses double (float64) to store and do its
      computation for hierarchical clustering. However, I have a very large
      dataset (292000x292000) that I would like to perform hclust on but my
      computer is RAM limited. I have 252GB RAM and I think the clustering
      algorithm should be able to work on my dataset when all values are stored
      and computed using float16s instead.</p>

      <p dir="auto">For large datasets on machines with insufficient RAM to
      store and compute on Arrays of float64s, it would be awesome if
      computation could be done on a different precision float to reduce the
      memory footprint.</p>

      <p dir="auto">Additionally, adding choices for datatypes could be very
      useful for almost all scipy functions.</p>

      <p dir="auto"><strong>Describe the solution you'd like</strong><br>

      Allow for an argument to specify what datatype you'd like to use (e.g.
      np.float64, np.float32, np.float16)</p>

      <p dir="auto">The argument could be like dtype='np.double' by default but
      changable to whatever datatype is chosen.</p>
    sentences:
      - >-
        <p dir="auto">One representative error:</p>

        <div class="snippet-clipboard-content notranslate position-relative
        overflow-auto"
        data-snippet-clipboard-copy-content="torch/csrc/autograd/functions/init.cpp:220:37:
        error: address of overloaded function 'getTupleAttr' does not match
        required type '_object *(_object *, void *)'
          {(char*)&quot;output_padding&quot;, (getter)getTupleAttr&lt;ConvBackwardBackward, std::vector&lt;int&gt;, ConvParams,
                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        torch/csrc/autograd/functions/init.cpp:82:11: note: candidate template
        ignored: invalid explicitly-specified argument for template parameter
        'Convert'

        PyObject* getTupleAttr(PyObject* obj, void* _unused)"><pre
        class="notranslate"><code
        class="notranslate">torch/csrc/autograd/functions/init.cpp:220:37:
        error: address of overloaded function 'getTupleAttr' does not match
        required type '_object *(_object *, void *)'
          {(char*)"output_padding", (getter)getTupleAttr&lt;ConvBackwardBackward, std::vector&lt;int&gt;, ConvParams,
                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        torch/csrc/autograd/functions/init.cpp:82:11: note: candidate template
        ignored: invalid explicitly-specified argument for template parameter
        'Convert'

        PyObject* getTupleAttr(PyObject* obj, void* _unused)

        </code></pre></div>

        <p dir="auto">The cause of the problem is <a class="commit-link"
        data-hovercard-type="commit"
        data-hovercard-url="https://github.com/pytorch/pytorch/commit/aa911939a328eff55c9b28b39ed3c43507ba8a2a/hovercard"
        href="https://github.com/pytorch/pytorch/commit/aa911939a328eff55c9b28b39ed3c43507ba8a2a"><tt>aa91193</tt></a>:</p>

        <div class="snippet-clipboard-content notranslate position-relative
        overflow-auto" data-snippet-clipboard-copy-content="  
        {(char*)&quot;output_padding&quot;, (getter)getTupleAttr&lt;ConvForward,
        std::vector&lt;int&gt;, ConvParams,

        -                                        
        &amp;ConvParams::output_padding, long, PyInt_FromLong&gt;, NULL, NULL,
        NULL},

        +                                        
        &amp;ConvParams::output_padding, int64_t, PyInt_FromLong&gt;, NULL,
        NULL, NULL},"><pre class="notranslate"><code class="notranslate">  
        {(char*)"output_padding", (getter)getTupleAttr&lt;ConvForward,
        std::vector&lt;int&gt;, ConvParams,

        -                                        
        &amp;ConvParams::output_padding, long, PyInt_FromLong&gt;, NULL, NULL,
        NULL},

        +                                        
        &amp;ConvParams::output_padding, int64_t, PyInt_FromLong&gt;, NULL,
        NULL, NULL},

        </code></pre></div>

        <p dir="auto">It seems that on clang, changing the type parameter here
        is sufficient to cause template instantiation to fail.</p>

        <p dir="auto">Maybe the easiest way to fix this is to write a more
        portable version of PyInt_FromLong (and friends) which always returns
        <code class="notranslate">int64_t</code>.</p>
      - >-
        <p dir="auto">I try the scipy ward clustering, when calculating linkage,
        it report follow error:</p>

        <div class="snippet-clipboard-content notranslate position-relative
        overflow-auto" data-snippet-clipboard-copy-content="ward_h = linkage(X,
        method='ward', metric='euclidean')

        Python(2557,0x7fff732cc310) malloc: ***
        mach_vm_map(size=18446744067627675648) failed (error code=3)

        *** error: can't allocate region

        *** set a breakpoint in malloc_error_break to debug

        ---------------------------------------------------------------------------

        MemoryError                               Traceback (most recent call
        last)

        &lt;ipython-input-10-769ae7c53f7c&gt; in &lt;module&gt;()

        ----&gt; 1 ward_h = linkage(X, method='ward', metric='euclidean')


        /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.pyc
        in linkage(y, method, metric)
            652             Z = np.zeros((n - 1, 4))
            653             _hierarchy_wrap.linkage_euclid_wrap(dm, Z, X, m, n,
        --&gt; 654                                              
        int(_cpy_euclid_methods[method]))
            655     return Z
            656 

        MemoryError: out of memory while computing linkage"><pre
        class="notranslate"><code class="notranslate">ward_h = linkage(X,
        method='ward', metric='euclidean')

        Python(2557,0x7fff732cc310) malloc: ***
        mach_vm_map(size=18446744067627675648) failed (error code=3)

        *** error: can't allocate region

        *** set a breakpoint in malloc_error_break to debug

        ---------------------------------------------------------------------------

        MemoryError                               Traceback (most recent call
        last)

        &lt;ipython-input-10-769ae7c53f7c&gt; in &lt;module&gt;()

        ----&gt; 1 ward_h = linkage(X, method='ward', metric='euclidean')


        /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.pyc
        in linkage(y, method, metric)
            652             Z = np.zeros((n - 1, 4))
            653             _hierarchy_wrap.linkage_euclid_wrap(dm, Z, X, m, n,
        --&gt; 654                                              
        int(_cpy_euclid_methods[method]))
            655     return Z
            656 

        MemoryError: out of memory while computing linkage

        </code></pre></div>

        <p dir="auto">How can I solve this？</p>

        <p dir="auto">The data set I use is here: <a
        href="https://dl.dropboxusercontent.com/u/68126956/df.csv"
        rel="nofollow">https://dl.dropboxusercontent.com/u/68126956/df.csv</a>.</p>

        <p dir="auto">Thanks.</p>
      - >-
        <p dir="auto">Make sure these boxes are checked before submitting your
        issue - thank you!</p>

        <ul dir="auto">

        <li>[yes ] I have checked the superset logs for python stacktraces and
        included it here as text if any</li>

        <li>[yes ] I have reproduced the issue with at least the latest released
        version of superset</li>

        <li>[yes ] I have checked the issue tracker for the same issue and I
        haven't found one similar</li>

        </ul>

        <h3 dir="auto">Superset version</h3>

        <p dir="auto">0.19.1</p>

        <h3 dir="auto">Expected results</h3>

        <p dir="auto">I try to draw mapbox in superset. I have dataset with
        column Latitude and Longitude and use it in respective field.</p>

        <h3 dir="auto">Actual results</h3>

        <p dir="auto">TypeError:
        &lt;superset.connectors.druid.models.DruidMetric object at 0xefbea90&gt;
        is not JSON serializable</p>

        <h3 dir="auto">Steps to reproduce</h3>

        <p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow"
        href="https://user-images.githubusercontent.com/13684386/31226202-8d89286c-a9d5-11e7-8d8b-eb7e9d6c4d77.png"><img
        src="https://user-images.githubusercontent.com/13684386/31226202-8d89286c-a9d5-11e7-8d8b-eb7e9d6c4d77.png"
        alt="togithub" style="max-width: 100%;"></a></p>

        <p dir="auto">Anyone  already have the same problem?<br>

        Thanks</p>
  - source_sentence: >-
      <p dir="auto">Challenge <a
      href="http://www.freecodecamp.com/challenges/waypoint-global-scope-and-functions#?solution=%2F%2F%20Declare%20your%20variable%20here%0Avar%20myGlobal%20%3D%2010%3B%0Avar%20oopsGlobal%3B%0A%0Afunction%20fun1%28%29%20%7B%0A%20%20%2F%2F%20Assign%205%20to%20oopsGlobal%20Here%0A%20%20oopsGlobal%20%3D%205%3B%0A%7D%0A%0A%2F%2F%20Only%20change%20code%20above%20this%20line%0Afunction%20fun2%28%29%20%7B%0A%20%20var%20output%20%3D%20%22%22%3B%0A%20%20if%28typeof%20myGlobal%20!%3D%20%22undefined%22%29%20%7B%0A%20%20%20%20output%20%2B%3D%20%22myGlobal%3A%20%22%20%2B%20myGlobal%3B%0A%20%20%7D%0A%20%20if%28typeof%20oopsGlobal%20!%3D%20%22undefined%22%29%20%7B%0A%20%20%20%20output%20%2B%3D%20%22%20oopsGlobal%3A%20%22%20%2B%20oopsGlobal%3B%0A%20%20%7D%0A%20%20console.log%28output%29%3B%0A%7D%0A"
      rel="nofollow">Waypoint: Global Scope and Functions</a> has an issue.<br>

      User Agent is: <code class="notranslate">Mozilla/5.0 (Macintosh; Intel Mac
      OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106
      Safari/537.36</code>.<br>

      Please describe how to reproduce this issue, and include links to
      screenshots if possible.</p>

      <p dir="auto">The test case has a typo in it:<br>

      Do not decalre oopsGlobal using the var keyword</p>
    sentences:
      - >-
        <p dir="auto">Challenge <a
        href="http://www.freecodecamp.com/challenges/waypoint-local-scope-and-functions#?solution=function%20myFunction%28%29%20%7B%0A%20%20var%20myVar%20%3D%205%3B%20%20%0A%20%20console.log%28myVar%29%3B%0A%7D%0AmyFunction%28%29%3B%0A%0A%2F%2F%20run%20and%20check%20the%20console%20%0A%2F%2F%20myVar%20is%20not%20defined%20outside%20of%20myFunction%0A%0A%2F%2F%20now%20remove%20the%20console.log%20line%20to%20pass%20the%20test%0A%0A"
        rel="nofollow">Waypoint: Local Scope and Functions</a> has an issue.<br>

        User Agent is: <code class="notranslate">Mozilla/5.0 (Macintosh; Intel
        Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko)
        Chrome/48.0.2564.48 Safari/537.36</code>.</p>

        <p dir="auto">The test asks you to remove the second console.log() to
        pass. When you remove the second console.log() it still fails. You have
        to remove both console.log() calls to pass(including the one inside
        myFunction().)</p>

        <p dir="auto">Reproducible every time by removing the console.log()
        outside of myFunction(), as asked by test. I'm assuming this is a bug in
        the test and not the text as there's no reason to remove the
        console.log() call within myFunction().</p>
      - >-
        <ul dir="auto">

        <li>VSCode Version: 0.10.11</li>

        <li>OS Version: Windows 7</li>

        </ul>

        <h2 dir="auto">Issue 1:</h2>

        <p dir="auto">In the following code (using C++ syntax highlighting), the
        string literal(s) are not highlighted at all.<br>

        This issue is exclusive to C++ syntax highlighting and does not occur
        with C syntax highlighting.</p>

        <h4 dir="auto">Code:</h4>

        <div class="snippet-clipboard-content notranslate position-relative
        overflow-auto" data-snippet-clipboard-copy-content="int main()

        {
            printf(&quot;%s\n&quot;, &quot;a string&quot;);
        }"><pre class="notranslate"><code class="notranslate">int main()

        {
            printf("%s\n", "a string");
        }

        </code></pre></div>

        <h4 dir="auto">Result</h4>

        <p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow"
        href="https://raw.githubusercontent.com/SE2Dev/VSCode-BugExample/master/resources/c-vs-cpp.png"><img
        src="https://raw.githubusercontent.com/SE2Dev/VSCode-BugExample/master/resources/c-vs-cpp.png"
        alt="Bug Screenshot" style="max-width: 100%;"></a></p>

        <h2 dir="auto">Issue 2:</h2>

        <p dir="auto">In custom tmLanguage based colorizers that reference
        source.c from within rules that begin with certain patterns, the inlined
        C highlighting is incorrect (this issue also applies to rules using
        source.c++ but does not occur in rules using source.js).</p>

        <h4 dir="auto">Code</h4>

        <p dir="auto">(See <a
        href="https://github.com/SE2Dev/VSCode-BugExample">https://github.com/SE2Dev/VSCode-BugExample</a>)</p>

        <h4 dir="auto">Result</h4>

        <p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow"
        href="https://raw.githubusercontent.com/SE2Dev/VSCode-BugExample/master/resources/screenshot.png"><img
        src="https://raw.githubusercontent.com/SE2Dev/VSCode-BugExample/master/resources/screenshot.png"
        alt="Bug Screenshot" style="max-width: 100%;"></a></p>
      - >-
        <p dir="auto">I'm not sure why the comments in the <code
        class="notranslate">variables.less</code> got changed to this style
        <code class="notranslate">//**</code>, but it my opinion it was a brave
        move.</p>

        <p dir="auto">For me at least it prevents compilation when using the
        nodejs compiler <code class="notranslate">lessc</code>.</p>

        <p dir="auto">At also breaks the syntax highlighting in my IDE, Visual
        Studio 2013</p>
  - source_sentence: He described his ambition as “a relentless pursuit of excellence.”
    sentences:
      - >-
        His goal is characterized by an unwavering drive toward high standards
        in every endeavor
      - Sidechain liquidity pools facilitate token swaps without mainnet fees
      - The harvest from her backyard was exceptionally crisp and vibrant
  - source_sentence: The honeybee's hum echoed over rows of flowering lavender
    sentences:
      - This cheese aged for months, developing deep amber tones
      - Between thunderclaps a lover's promise rang clear
      - >-
        Thoughtful acts, no matter how modest, have the power to lift spirits
        and create positive ripple effects
  - source_sentence: Two men working on their knees in front of a building.
    sentences:
      - There are people working.
      - A man is in a kayak.
      - There are people not working.
datasets:
  - aisuko/quora_duplicate_questions
pipeline_tag: sentence-similarity
library_name: sentence-transformers

SimpleEmbed

This is a sentence-transformers model trained on the hanzceo/sts-en-en, jinaai/negation-dataset, andersonbcdefg/jina_negation_v2, WhereIsAI/github-issue-similarity and aisuko/quora_duplicate_questions datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Maximum Sequence Length: inf tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text
Training Datasets:
- hanzceo/sts-en-en
- jinaai/negation-dataset
- andersonbcdefg/jina_negation_v2
- WhereIsAI/github-issue-similarity
- aisuko/quora_duplicate_questions
Language: en
License: mit

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): StaticEmbedding({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("hanzceo/SimpleEmbed-dev1")
# Run inference
queries = [
    'Two men working on their knees in front of a building.',
]
documents = [
    'There are people working.',
    'There are people not working.',
    'A man is in a kayak.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.4950,  0.1923, -0.0330]])

Training Details

Training Datasets

hanzceo/sts-en-en

Dataset: hanzceo/sts-en-en
Size: 1,182 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 100 samples:

	anchor	positive	negative
type	string	string	string
modality	text	text	text
details	min: 60 characters mean: 79.23 characters max: 107 characters	min: 59 characters mean: 75.64 characters max: 112 characters	min: 41 characters mean: 72.0 characters max: 98 characters

Samples:

anchor	positive	negative
`The intricate framework elucidates previously uncharted dimensions of cellular plasticity`	`Researchers have unveiled an innovative approach that captures dynamic fluctuations within cellular architecture`	`The old wooden bridge creaked under the weight of the passing tractor`
`Observers note a pronounced correlation between epigenetic markers and metabolic fluxes`	`Analysts discovered subtle shifts in transcriptional patterns across divergent cell populations`	`The stock market experienced a slight dip following the unexpected jobs report`
`The novel assay integrates multi-omics datasets to reconstruct regulatory networks`	`Investigators applied machine learning to synthesize heterogeneous biological information`	`Apples float in water because they are made up of 25 percent air`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

jinaai/negation-dataset

Dataset: jinaai/negation-dataset
Size: 10,000 training samples
Columns: anchor, entailment, and negative

Approximate statistics based on the first 100 samples:

	anchor	entailment	negative
type	string	string	string
modality	text	text	text
details	min: 25 characters mean: 68.11 characters max: 189 characters	min: 14 characters mean: 35.34 characters max: 107 characters	min: 16 characters mean: 38.9 characters max: 105 characters

Samples:

anchor	entailment	negative
`Two young girls are playing outside in a non-urban environment.`	`Two girls are playing outside.`	`Two girls are not playing outside.`
`A man with a red shirt is watching another man who is standing on top of a attached cart filled to the top.`	`A man is standing on top of a cart.`	`A man is not standing on top of a cart.`
`A man in a blue shirt driving a Segway type vehicle.`	`A person is riding a motorized vehicle.`	`A person is not riding a motorized vehicle.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

andersonbcdefg/jina_negation_v2

Dataset: andersonbcdefg/jina_negation_v2
Size: 50,000 training samples
Columns: query, pos, and neg

Approximate statistics based on the first 100 samples:

	query	pos	neg
type	string	string	string
modality	text	text	text
details	min: 33 characters mean: 103.12 characters max: 309 characters	min: 20 characters mean: 58.65 characters max: 164 characters	min: 22 characters mean: 58.88 characters max: 156 characters

Samples:

query	pos	neg
`A dog happily looking onward in the back seat of a car.`	`A dog sitting down inside of a vehicle.`	`A dog standing up outside of a vehicle.`
`Alas, madame, said Poirot, "I thought you had come to honour me with a visit!"`	`Poirot said that he thought the woman had come to visit him.`	`Poirot said that he thought the woman had not come to visit him.`
`yeah i i i agree i the thing that scares me uh though about where i would i would want definitely want some sort of legislation and coming from the north east i'm just not used to seeing um these and i i know this may sound kind of stereo typical but the cowboys with the gun racks in the back of their trucks`	`I would want the law to cover guns.`	`I would not want the law to cover guns.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

WhereIsAI/github-issue-similarity

Dataset: WhereIsAI/github-issue-similarity
Size: 9,457 training samples
Columns: anchor and positive

Approximate statistics based on the first 100 samples:

	anchor	positive
type	string	string
modality	text	text
details	min: 0 characters mean: 4993.44 characters max: 135575 characters	min: 0 characters mean: 3329.07 characters max: 19882 characters

Samples:

anchor	positive
`Bug summary I put a torch.Tensor in matplotlib.pyplot.hist() , but it draw a wrong graphic and take a long time. Although transform to numpy, the function work well. But all the others function I used are work well on tensor. So I think its a bug. Code for reproduction` `import matplotlib.pyplot as plt import torch plt.hist(torch.randn...`	`Bug report Bug summary Generating np.random.randn(1000) values, visualizing them with plt.hist(). Works fine with Numpy. When I replace Numpy with tensorflow.experimental.numpy, Matplotlib 3.3.4 fails to display the histogram correctly. Matplotlib 3.2.2 works fine. Code for reproduction import matplotlib.`
`https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/container_bridge.go#L122-L143 container_bridge.go assumes that the virtual IP of services & pods will be in the 10. space. I propose there is no reason to make this assumption. As outlined in #15932, cluster admins may need to deploy to hosts in which 10. is reserved for the nodes. In such a case, Kubelets must support an alternative range.`	`Today kubelet sets up an iptables MASQUERADE rule for any traffic destined for anything except 10.0.0.0/8. This is close, but not even correct on GCE, and certainly not right elsewhere. First GCE. We probably want something like: iptables -t nat -N KUBE-IPMASQ iptables -t nat -A KUBE-IPMASQ -d 10.0.0.0/8 -j RETURN iptables -t nat -A KUBE-IPMASQ -d 172.16.0.0/12 -j RETURN iptables -t nat -A KUBE-IPMASQ -d 192.168.0.0/16 -j RETURN iptables -t nat -A KUBE-IPMASQ -j MASQUERADE iptables -t nat -I...`
`Is there an existing issue for this? I have searched the existing issues This issue exists in the latest npm version I am using the latest npm Current Behavior Currently, my package.json specifies "typescript": "^5.0.2". When I change it to say "typescript": "^5.0.3", npm 9 spins for 4:28 before deciding it doesn't exist. For comparison, npm 8 installs it with no problem in 0:44. Ironically, I can't upgrade npm to 9.6 due to this issue: npm 9.5.1 times out when I run npm i -g npm.` `Expec...`	Is there an existing issue for this? I have searched the existing issues This issue exists in the latest npm version I am using the latest npm Current Behavior When running npm install it will sometimes hang at a random point. When it does this, it is stuck forever. CTRL+C will do nothing the first time that combination is pressed when this has occurred. Pressing that key combination the second time will make the current line (the one showing the little progress bar) disappear but that's it. No further responses to that key combination are observed. `The C...`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

aisuko/quora_duplicate_questions

Dataset: aisuko/quora_duplicate_questions at a14d279
Size: 149,263 training samples
Columns: question1 and question2

Approximate statistics based on the first 100 samples:

	question1	question2
type	string	string
modality	text	text
details	min: 16 characters mean: 54.75 characters max: 139 characters	min: 21 characters mean: 54.34 characters max: 127 characters

Samples:

question1	question2
`Astrology: I am a Capricorn Sun Cap moon and cap rising...what does that say about me?`	`I'm a triple Capricorn (Sun, Moon and ascendant in Capricorn) What does this say about me?`
`How can I be a good geologist?`	`What should I do to be a great geologist?`
`How do I read and find my YouTube comments?`	`How can I see all my Youtube comments?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        512,
        256,
        128,
        64,
        32
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 256
learning_rate: 0.002
weight_decay: 0.01
num_train_epochs: 10
warmup_ratio: 0.1
seed: 69
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
prediction_loss_only: True
per_device_train_batch_size: 256
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 0.002
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 69
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0012	1	7.9156
1.1601	1000	11.1122
2.3202	2000	6.1974
3.4803	3000	4.1201
4.6404	4000	3.2187
5.8005	5000	2.6037
6.9606	6000	2.2084
8.1206	7000	1.887
9.2807	8000	1.8408

Training Time

Training: 37.3 minutes

Framework Versions

Python: 3.12.6
Sentence Transformers: 5.6.0
Transformers: 4.56.0
PyTorch: 2.8.0+cu129
Accelerate: 1.10.1
Datasets: 5.0.0
Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

hanzceo
/

SimpleEmbed-dev1

SimpleEmbed

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Datasets

hanzceo/sts-en-en

jinaai/negation-dataset

andersonbcdefg/jina_negation_v2

WhereIsAI/github-issue-similarity

Bug summary

Code for reproduction

Bug report

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

`Expec...`

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

aisuko/quora_duplicate_questions

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Training Time

Framework Versions

Citation

BibTeX

Sentence Transformers

MatryoshkaLoss

MultipleNegativesRankingLoss