After 2 months of refinement, I'm happy to announce that a lot of Transformers' modeling code is now significantly more torch-compile & export-friendly π₯
Why it had to be done π PyTorch's Dynamo compiler is increasingly becoming the default interoperability layer for ML systems. Anything that relies on torch.export or torch.compile, from model optimization to cross-framework integrations, benefits directly when models can be captured as a single dynamo-traced graph !
Transformers models are now easier to: βοΈ Compile end-to-end with torch.compile backends π¦ Export reliably via torch.export and torch.onnx.export π Deploy to ONNX / ONNX Runtime, Intel Corporation's OpenVINO, NVIDIA AutoDeploy (TRT-LLM), AMD's Quark, Meta's Executorch and more hardware-specific runtimes.
This work aims at unblocking entire TorchDynamo-based toolchains that rely on exporting Transformers across runtimes and accelerators.
We are doubling down on Transformers commitment to be a first-class citizen of the PyTorch ecosystem, more exportable, more optimizable, and easier to deploy everywhere.
There are definitely some edge-cases that we still haven't addressed so don't hesitate to try compiling / exporting your favorite transformers and to open issues / PRs.
PR in the comments ! More updates coming coming soon !
I just released omarkamali/wikipedia-labels, with all the structural labels and namespace from wikipedia in 300+ languages. A gift for the data preprocessors and cleaners among us.
The concept of AI agentsβcombining models, tools, and orchestrationβhas become fairly standardized during the last year, but VLAgentIc brings something unique:
- Agents communicate over XMPP, enabling concurrent tasks and asynchronous messaging thanks to the SPADE framework. - Built-in presence and discovery streamline interactions between components. - Flexible behaviours make orchestrating AI-assisted security workflows seamless for future connections - Last but not least, the VLAI Severity and VLAI CWE classifiers are now wrapped as LLM Tools and run entirely locally.
New, more comprehensive agent tools will soon be available, leveraging the Vulnerability-Lookup API and supporting the GCVE project.
The Human-in-the-Loop agent tool will be designed to notify you and request authorization whenever a query to an external service is about to be madeβensuring that, by default, all reasoning and processing stay local on your computer.
Introducing the Z Image Turbo LoRA DLC App, a gallery space for plug-and-play Z-Image-Turbo LoRAs. It features a curated collection of impressive LoRAs for generating high-quality images. By default, it runs on the base model. Simply choose a LoRA, type your prompt, and generate images. You can find the app and more details below. π€π§ͺ
We now have a Deep Research for academia: SurveyX automatically writes academic surveys nearly indistinguishable from human-written ones π₯
Researchers from Beijing and Shanghai just published the first application of a deep research system to academia: their algorithm, given a question, can give you a survey of all papers on the subject.
To make a research survey, you generally follow two steps, preparation (collect and organize papers) and writing (outline creation, writing, polishing). Researchers followed the same two steps and automated them.
π― For the preparation part, a key part is find all the important references on the given subject. Researchers first cast a wide net of all relevant papers. But then finding the really important ones is like distilling knowledge from a haystack of information. To solve this challenge, they built an βAttributeTreeβ object that structures key information from citations. Ablating these AttributeTrees significantly decreased structure and synthesis scores, so they were really useful!
π For the writing part, key was to get a synthesis that's both short and true. This is not easy to get with LLMs! So they used methods like LLM-based deduplication to shorten the too verbose listings made by LLMs, and RAG to grab original quotes instead of made-up ones.
As a result, their system outperforms previous approaches by far!
As assessed by LLM-judges, the quality score os SurveyX even approaches this of human experts, with 4.59/5 vs 4.75/5 π
