new

Get trending papers in your email inbox!

Subscribe

Daily Papers

byAK and the research community

Oct 21

The ETH matrix model for DSSYK: non-perturbative corrections and intersection theory

At leading order in the genus expansion the ETH matrix model for DSSYK reproduces its correlators by construction, while its higher-genus corrections are conjectured to capture higher-topology contributions in the dual sine-dilaton gravity -- a correspondence established so far only for the disk and the wormhole. At fixed genus the correlators are built from discrete volumes N_{g,n}, polynomial in q-deformed zeta values ζ_q(2k) with q=e^{-λ}, λ being the DSSYK coupling. These lie in the ring of quasimodular forms generated by the Eisenstein series E_2,E_4,E_6, whose S-duality yields an exact closed form for the leading non-perturbative correction as λto0, controlled by widetilde q=e^{-4π^2/λ}. Known at disk level, this scale is shown here to govern the fixed-genus, higher-boundary amplitudes as well. We show that the term linear in widetilde q, at leading order in λ, is captured entirely by the q-deformed Weil--Petersson volumes, and reduces to a finite sum of intersection numbers of κ-classes on the moduli space mathcal M_{g,n} of stable curves, computable without repeating the topological recursion that produced the N_{g,n}. We tabulate it for every (g,n) whose q-deformed volume is known in closed form, and extend it to (3,1),(3,2),(4,1), where none is available. The construction is not restricted to leading order: we work out O(widetilde q^2) for the same cases.

  • 2 authors
·
Aug 23

Emergent Mixture-of-Experts: Can Dense Pre-trained Transformers Benefit from Emergent Modular Structures?

Incorporating modular designs into neural networks demonstrates superior out-of-generalization, learning efficiency, etc. Existing modular neural networks are generally explicit because their modular architectures are pre-defined, and individual modules are expected to implement distinct functions. Conversely, recent works reveal that there exist implicit modular structures in standard pre-trained transformers, namely Emergent Modularity. They indicate that such modular structures exhibit during the early pre-training phase and are totally spontaneous. However, most transformers are still treated as monolithic models with their modular natures underutilized. Therefore, given the excellent properties of explicit modular architecture, we explore whether and how dense pre-trained transformers can benefit from emergent modular structures. To study this question, we construct Emergent Mixture-of-Experts (EMoE). Without introducing additional parameters, EMoE can be seen as the modular counterpart of the original model and can be effortlessly incorporated into downstream tuning. Extensive experiments (we tune 1785 models) on various downstream tasks (vision and language) and models (22M to1.5B) demonstrate that EMoE effectively boosts in-domain and out-of-domain generalization abilities. Further analysis and ablation study suggest that EMoE mitigates negative knowledge transfer and is robust to various configurations. Code is available at https://github.com/qiuzh20/EMoE

  • 3 authors
·
Oct 16, 2023