Commit History
-
Conversation trees (#223) (#807)
Unverified signature* work on branching * branching * work on input field * wip * wip * helper stuff * pass tests * add type guards and clean up type requirements * fixed shared conv type * more helpers stuff * clean up code * addChildren helper * addSibling * add test for addSibling & refacto addChildren tests * backend work pt. 1 * backend done * add children property to messages for easier frontend rendering * fix message id type rail * front-end work on simple linear conversation * fix title generation * convert conversations on post * server side retry works with branching * clean up buttons * fix retry feature backend * Send new messages in any subtree * make edit previous prompts feature work * fix padding * revert unneeded changes * bring back pending message * fix front-end streaming * fix initial message * Revert "fix initial message" This reverts commit 6257fe8789c52a20e7506aa453efda55c16e95bb. * …e6addfc
-
Added support for SearXNG as a websearch source (#805)
Karlostavitch1 +1: nsarrazin Unverified signature* Created searchSearxng.ts for querying and parsing searxng instance. Added searxng to the searchWeb.ts to ensure inclusion in the search initialisation script. Added searxng to +layout.server.ts to ensure the search button is enabled if url provided in .env.local assignment Updated .env to include searxng configuration Updated README.md under Web Search Config to mention addition of searxng URL use in .env * Changes requested consolidated env variable to SEARXNG_QUERY_URL (felt considering the content the word QUERY was more relevant) refactored the searchSearxng.ts to generate the query with the new variable updated related references to variable (.env, searchWeb.ts and +layout.server.ts) Updated readme and variable comment on .env to reflect the same * urefectored to display SearXNG in search dialoge when used. * lint * types --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>7c22da3
-
Web search: UI update (#749)
Unverified signature* wip * ui * font-size * leading * smaller * move icon and remove loading props * rm inline svg * rm animationbccd811
-
[Fix] small fixes in websearch generateQuery (#727)
MishigUnverified signature13489e8
-
[JS] use shorthand notation (#711)
MishigUnverified signature* [JS] use shorthand notation * add `"object-shorthand": ["error", "always"],` * fix "Expected method shorthand object-shorthand" * formatd5559df
-
Add embedding models configurable, from both transformers.js and TEI (#646)
Unverified signature* Add embedding models configurable, from both Xenova and TEI * fix lint and format * Fix bug in sentenceSimilarity * Batches for TEI using /info route * Fix web search disapear when finish searching * Fix lint and format * Add more options for better embedding model usage * Fixing CR issues * Fix websearch disapear in later PR * Fix lint * Fix more minor code CR * Valiadate embeddingModelName field in model config * Add embeddingModel into shared conversation * Fix lint and format * Add default embedding model, and more readme explanation * Fix minor embedding model readme detailed * Update settings.json * Update README.md Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Update README.md Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Apply suggestions from code review Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Resolved more issues * lint * Fix more issues …3a01622
-
Fixes "invalid url" error in websearch (#675)
Unverified signature* Fixed #656 * lint --------- Co-authored-by: muhammadbaqir1327 <muhammad.baqir@arbisoft.com> Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>e63c1d7 -
Use JSON5 for parsing all config vars (#671)
Unverified signaturef09c5f3
-
Add support for SerpStack API (#669)
Unverified signature* Add support for SerpStack API * update const to SERPSTACK_API_KEY * update readme * lint --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>f02ffb2
-
Add allow & blocklist for websearch (#651)
Unverified signature* Add allow & blocklist for websearch * fix start whitespacebdb1788
-
Option for running websearch locally (#563)
Unverified signature* Fully local websearch * better comments * fix else ife3af794
-
Enable lazy construction of pipeline (#554)
Joshua LochnerUnverified signature683ef2f -
Modular backends & support for openAI & AWS endpoints (#541)
Unverified signature* Fix the response Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Should use /completions Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Use async generator Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Use openai npm Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Fix generateFromDefaultEndpoint Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Fix last char become undefined Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Better support for system prompt Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Updates Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Revert Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Update README Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Default system prompt Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * remove sk- Signed-off-by: Hung-Han …9db8ced
-
add you.com integration (#514)
mrbean +2: David Li, MishigUnverified signature* add you.com integration * [Feat] Add types (#1) * feat: add types * feat: specify search provider * [Feat] add return values (#3) * feat: add types * feat: specify search provider * add values for ui * add idea to gitignore * lint and downgrade * Revert "lint and downgrade" This reverts commit fbfe012cf4d6aaf5ea00508ca0f99a408ce152e6. * Updates to you.com integration (#4) * `npm run format` * enable search when `YDC_API_KEY` is available * typing nit * enum WebSearchProvider * eslint-disable-next-line no-shadow * downgrade * You.com already provides text snippets (#5) * Order desc you.com results by text length * You.com already provides texts from webpages * add to env * Update .env Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Update src/lib/server/websearch/searchWeb.ts --------- Co-authored-by: David Li <david.li3100@gmail.com> Co-authored-by: Mishig <mishig.davaadorj@coloradocoll…4606755
-
[Websearch] UI fix sources (#521)
MishigUnverified signature5071731
-
Update websearch prompting & summary prompting (#503)
Unverified signature* Use the task model for websearch query generation * summary with examples in chat format * Update to websearch prompting - Removed web search template - Added examples in `generateQuery` - Passed previous questions when generating the web search query - Added previous questions to the bottom of the context window in `buildPrompt` when using websearch * wait for model if needed * rm console.log * get rid of regex * added a few time related examples * Update src/lib/server/websearch/generateQuery.ts Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * reformat yesterday * test * fix whitespace * remove extra whitespace for mistral * removed mistral extra whitespace * Update src/lib/server/summarize.ts Co-authored-by: Victor Muštar <victor.mustar@gmail.com> * Update src/lib/server/summarize.ts Co-authored-by: Victor Muštar <victor.mustar@gmail.com> * add an emoji if none is found in the first three characters ----…1e5090f
-
remove logs for parsing
140086b
-
Make embedding model settings more future-proof (#454)
MishigUnverified signature3acc11d -
Backend refactor for streaming endpoints (#444)
Unverified signature* wip: complete refactor of streaming backend * working refactoring * fix missing first token & perf regression in output quality * lint * Fix websearch loading from db * fix loading * fix invalidate * remove logs * fix SSR error * typo: paragraphs * fixed save on abort * lint * lint * remove debug log in console * lint for reale943a05 -
Revert "Revert "Update embedding model for WebSearch (#437)""
This reverts commit 1061bc237c37ada9537197b10aeee43e08fc8f4e.aa07e29 -
Revert "Update embedding model for WebSearch (#437)"
This reverts commit f88542b1121903c1df34b20bd7b8213148e85ca0.1061bc2 -
Update embedding model for WebSearch (#437)
MishigUnverified signature* Use `gte-base` as the emebdding model * use `bge-small-en-v1.5` * Revert "use `bge-small-en-v1.5`" This reverts commit 8cfe084d26884c130d6034afa095b7bbf0f7fa1b. * Use `gte-small`f88542b
-
[Websearch] update (#427)
Mishig +2: julien-c, victor Unverified signature* Fix reuqest body * update webSearchQueryPromptTemplate * update generate google query parser * Add today's date to google search query creator * crawl top stories if exts; remove answer_box & knowledgeGraph * Create paragraph chunks from top articles * flattened paragprah chunks * update status texts * add gradio client * call gradio app for RAG * Web scrape only "p, li, span" els * add MAX_N_CHUNKS * gradio result typing * parse only <p> elements * rm dev change * update typing WebSearch * buld RAG prompt * Rm dev change * change websearch context msg from user to assisntat type * use hosted gradio app * fix lint * prompt engineering * more prompt engineering * MAX_N_PAGES_SCRAPE = 10 * better error msg * more prompt engineering * revert websearch prompt to previous * rm `top_stories` from websearch as the results are not good * Stop using gradio client, use regular fetch * chore * Rm websearch…ebac87f
-
Make all prompt templates configurable (#400)
Unverified signature* Make all prompt templates configurable This PR enables all prompts to be configurable using handlebar templates as described in #382. For backward compatibility the old hardcoded templates are reimplemented as the default templates. The old template parameters such as `preprompt`, `userMessageToken`, `userMessageEndToken`, `assistantMessageToken`, `assistantMessageEndToken` are now considered legacy. They still work as they are exposed as variables to the default template. However, new prompt configurations should not use these. And it is recommended that the legacy support is eventually removed. As an example, this is how the default chat prompt template is implemented: ``` {{preprompt}} {{#each messages}} {{#ifUser}}{{@root.userMessageToken}}{{content}}{{@root.userMessageEndToken}}{{/ifUser}} {{#ifAssistant}}{{@root.assistantMessageToken}}{{content}}{{@root.assistantMessageEndToken}}{{/ifAssistant}} {{/each}} {{assistantMessageToken}} ``` In addition, …447c0ca
-
allow different user and assistant end-token (#375)
Unverified signature* allow different user and assistant end-token For models like Llama2, the EndToken is not the same for a userMessage and an assistantMessage. This implements `userMessageEndToken` and `assistantMessageEndToken` which overwrites the messageEndToken behavior. This PR also allows empty strings as userMessageToken and assistantMessageToken and makes this the default. This adds additional flexibility, which is required in the case of Llama2 where the first userMessage is effectively different because of the system message. Note that because `userMessageEndToken` and `assistantMessageToken` are nearly always concatenated, it is almost redundant to have both. The exception is `generateQuery` for websearch which have several consecutive user messages. * Make model branding customizable based on env var (#345) * rm open assistant branding * Update SettingsModal.svelte * make settings work with a dynamic list of models * fixed types --------- Co-authored-…f209301
-
Add support for HF summarization endpoint in the websearch (#319)
Unverified signature* Add support for HF endpoint for summary * add fail-safe for summarization10d1ab5
-
Added Serper.dev API as an alternative web search provider (#302)
rogerserperUnverified signature6f7b315
-
Refactor of websearch (#281)
Unverified signature* broke up websearch into multiple endpoints * Refactored loading to use the load function instead of client side fetching * lint * Chat Logo Home Screen Bookmark icons for iOS (#279) * prettier fix for #279 * fix eslint --------- Co-authored-by: Carolyn Marie <99927612+CarolynM8ri3@users.noreply.github.com>6434339