sentence-transformers How to use Pravallika6/cross-domain-embeddings with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Pravallika6/cross-domain-embeddings")
sentences = [
"Can someone explain how to model dependencies in relational information?",
"[Preamble]\nRodr´ ıguez et al.\nGeneralized Boozer coordinates: a natural coordinate system for\nquasisymmetry\nE. Rodr ´ ıguez,1, 2,a) W. Sengupta,1, 2 and A. Bhattacharjee 1, 2\n1)Department of Astrophysical Sciences, Princeton University, Princeton, NJ,\n\n2)Princeton Plasma Physics Laboratory, Princeton, NJ, 08540\n(Dated: 22 September 2021)\nWe prove the existence of a straight-field-line coordinate system we call generalized Boozer coordinates. This\ncoordinate system exists for magnetic fields with nested toroidal flux surfaces provided\n∮\ndl/B(j ·∇ψ) = 0,\nwhere symbols have their usual meaning, and the integral is taken along closed magnetic field lines. All\nquasisymmetric fields, regardless of their associated form of equilibria, must satisfy this condition. This\ncoordinate system presents itself as a convenient form in which to describe general quasisymmetric configurations and their properties. Insight can be gained analytically into the difference between strong and weak\nforms of quasisymmetry, as well as axisymmetry, and the interaction of quasisymmetry with different forms\nof equilibria.\nI. INTRODUCTION:\nQuasisymmetric stellarators1–3 (sometimes referred to\nas helically-symmetric4,5) constitute an attractive choice\nfor magnetic confinement fusion. Theoretically, such\ndesigns exhibit transport properties analogous to those\nof axisymmetric devices 6 while possessing larger threedimensional freedom. The latter allows avoiding some\nof the inherent limitations of tokamaks. The quasisymmetric stellarator achieves this by possessing a hidden\nsymmetry, namely, the magnitude of the magnetic field\n|B|is symmetric, while the full vector B is not.\nThe concept of quasisymmetry (QS) is elegant, but\nit appears to have a significant theoretical limitation. It\nwas soon realized7 that constructing a configuration with\nexact QS is not possible. Although there is no definitive\nproof that shows this impossibility, work on near-axis\nexpansions7,8 supports this point of view. The governing\nsystem of equations is overdetermined: there are more\nconstraints than degrees of freedom. This limitation does\nnot, however, prevent designs that exhibit behavior close\nto QS in a volume. 9,10\nRecently, studies that explore the concept of QS more\ndeeply have appeared.3,11 The main idea has been to separate the concept of QS from that of macroscopic force\nbalance. In these studies, QS is defined as a property of\nthe configuration that confers the single-particle dynamics an approximately conserved quantity without making any statement about the form of macroscopic equilibrium. This perspective differs significantly from the\nstandard approach. Prior to [3] and [12] (and with very\nfew exceptions5,13), the concept of QS was framed in the\ncontext of magnetohydrostatic (MHS) equilibria satisfying j ×B = ∇p, where j = ∇×B is the current density\nand p is the scalar pressure. As MHS is not intrinsic to\na)Author to whom correspondence should be addressed: eduardor@princeton.edu\nQS, it is important to define QS without reference to a\nparticular form of equilibrium. Separating QS from equilibrium allows us to investigate more deeply its meaning\nand limitations.\nAbandoning the convenient form of MHS equilibrium,\nalthough conceptually appropriate, comes at a cost. The\nmagnetic field no longer needs to satisfy the condition j ·∇p = 0 , implicitly assumed in most of the\nliterature1,4,6. Hence, one cannot guarantee the existence\nof Boozer coordinates 2 as presently understood, even if\nmagnetic flux surfaces exist. Boozer coordinates are particularly convenient for studying QS, as it presents the\nsymmetry in an explicit, simple form. This leads to a\nsearch for an analogous, convenient, but more general\ncoordinate system for quasisymmetric configurations.\nIn this paper, we construct such a coordinate system,\nwhich we call generalized Boozer coordinates (GBC).\nThis coordinate system was used to formulate the nearaxis expansion in [14], in which a QS equilibrium with\nanisotropic pressure was shown to avoid the conventional\nproblem of overdetermination. The present paper is organized as follows. First, we introduce, develop and discuss\nthis coordinate system systematically and rigorously. We\nstart by presenting a constructive proof for GBC and the\nclass of fields for which it exists. We then present the set\nof equations describing a quasisymmetric magnetic field\nin this coordinate system. This gives us the opportunity\nto gain an alternative 3,12 perspective on the distinction\nbetween the so-called weak and strong forms of quasisymmetry, as well as a comparison to axisymmetry. We close\nby summarizing the equations that link the equilibrium\nand the quasisymmetric field and concluding remarks.\nII. GENERALIZED BOOZER COORDINATES\nA. Explicitly symmetric formulation of quasisymmetry\nLet us start this section by introducing the notion of\nQS. We consider QS from the recent general perspective\narXiv:2106.08266v2 [physics.plasm-ph] 21 Sep 2021\nRodr´ ıguez et al. 2\nof single particle motion. 3,11 QS (and in particular weak\nQS) is the property of the fields that confers the motion\nof guiding-centre single particles with an approximately\nconserved quantity.3 For the dynamics of particles to exhibit this conservation it is necessary for the magnetic\nfield to have nested flux surfaces B ·∇ψ= 0 and satisfy,\n∇ψ×∇B·∇(B ·∇B) = 0. (1)\nThis form of quasisymmetry is most commonly referred\nto as the triple vector product formulation. Although\nthis is not the form that comes directly from the singleparticle analysis (that form is the one used in Sec. IIIA),\nit is a succinct way to impose QS on a magnetic field.\nGiven that this generalized concept of QS has a single\nparticle origin, no notion of macroscopic equilibrium is\ninvolved. Of course, from a practical point of view, any\nsteady field of interest will be in some form of force balance. With the definition of QS in (1), different forms of\nequilibria may be investigated to understand how they\ninteract with QS. One may generally refer to the macroscopic forces by F, and we do not attempt to assess their\norigin in this paper. Instead, we focus on the requirements at a fluid level. A complete view of the problem would require an investigation of the kinetic basis\nof the forces to link the fluid forces to microscopic behavior. This is particularly important as microscopic\nforces could break the symmetry (and with it, the QSrelated conserved momenta). An important example is\nthe electrostatic potential, whose symmetry is needed to\nthe appropriate order and imposes constraints even on\nthe forces that arise from the plasma, such as centrifugal\nforce or anisotropic pressure. With this in mind, we shall\nfocus on the macroscopic aspects.\nLooking at the statement of QS in the form presented\nin Eq. (1), the existence of a symmetry in the problem\nis not apparent. In the usual context of j ×B = ∇p,\nthis absence of obvious symmetry can be amended by\nusing Boozer coordinates. In these coordinates, a field in\nmagnetostatic equilibrium with well-defined flux surfaces\nis quasisymmetric (aside from quasipoloidal symmetry)\niff,\nB = B(ψ,θ −˜αφ), (2)\nwhere {ψ,θ,φ }are Boozer coordinates, ψthe flux surface\nlabel, θ and φ poloidal and toroidal angles respectively,\nand ˜α = N/M|M,N ∈Z. In order to employ Boozer\ncoordinates, the j·∇ψ= 0 property of MHS equilibria is\ncentral. Boozer coordinates have the standard straight\nfield-line coordinate Jacobian\nJ= Bφ + ιBθ\nB2 , (3)\nwhere the covariant Bθ and Bφ are flux functions,\nB = Bθ(ψ)∇θ+ Bφ(ψ)∇φ+ Bψ∇ψ.\nBoozer coordinates are widely used in stellarator theory\nand applications. The coordinates are a natural set that\nsimplify many of the governing equations, including QS.\nIn particular, construction of solutions by near-axis expansion of three-dimensional fields is most convenient in\nthese coordinates7,8,14–16.\nIn the context of our general quasisymmetric B, it is\nnot necessary to assume that j ·∇ψ = 0. However, we\ndemonstrate that any quasisymmetric field must satisfy∮\n(dl/B)(j ·∇ψ) = 0 (see Appendix A). The question\nthat naturally arises is, can we construct an appropriate straight field line coordinate system that explicitly\nexpresses the symmetry in QS in the Boozer fashion?\nThe answer is yes. To see this, we write Eq. (1) in\nstraight-field line coordinates {ψ,θ,φ } using the contravariant representation B = ∇ψ×(∇θ−ι∇φ),\n(∇ψ×∇θ∂θB+ ∇ψ×∇φ∂φB)·\n·∇(J−1(∂φB+ ι∂θB)) = 0,\nwhere J is the Jacobian associated to the coordinate\nsystem chosen. Now assume we can construct a straightfield line coordinate system with a Jacobian of the form\nJ= J(ψ,B), then the Jacobian factor may be dropped\nfrom the equation above to obtain,\n(∇ψ×∇θ∂θB+ ∇ψ×∇φ∂φB) ·∇(∂φB+ ι∂θB) = 0\ni.e.\n[\n∂φ −\n(∂φB\n∂θB\n)\n∂θ\n]\n(∂φ + ι∂θ)B = 0\nassuming that ∂θB ̸= 0. (This makes quasi-poloidally\nsymmetric solutions a special case.) From near-axis expansion we know that these solutions have a very restricted QS region3.\nCommuting operators, we obtain\n(∂φ + ι∂θ)\n(∂φB\n∂θB\n)\n∂θB = (b ·∇)\n(∂φB\n∂θB\n)\n∂θB = 0\nwhich implies that B = B(ψ,θ −˜αφ) or B = B(ψ,φ),\nwhere ˜α= −∂φB/∂θB is a flux function. The additional\nrequirement that ˜α is rational to avoid B = B(ψ) at\nnon-degenerate surfaces requires ˜α to be constant.3\nIn summary, if we are able to construct a straight field\nline coordinate system that has a Jacobian that can be\nwritten as a function of ψ and B only, then a field is QS\nin the weak sense if and only if B depends on a single\nlinear combination of those coordinate angles. Note that\nunder this assumption, the reverse direction of the proof\nis straightforward.\nB. Constructing generalized Boozer coordinates\nWe define generalized Boozer coordinates(GBC) as a\nset of straight field line coordinates whose Jacobian can\nbe written in the form J= Bα(ψ)/B2, where Bα is some\nflux function without requiring j ·∇ψ to vanish identically. This choice is more general than what is permitted\nby Boozer coordinates, which separately requires Bθ and\nRodr´ ıguez et al. 3\nBφ to be flux functions. We shall now constructively explore the conditions under which such a coordinate system exists.\nLet us start with a given magnetic field, assuming that\nit lies on well-defined flux surfaces. It then follows 2,17,18\nthat there exists some initial straight field line coordinate\nsystem {ψ,θ,φ }, so that B = ∇ψ×∇θ+ι∇φ×∇ψ. The\ndefinition of the angular straight field line coordinates is\narbitrary up to a transformation of the form\nθ= θ′+ ιω,\nφ= φ′+ ω.\nThe function ω = ω(ψ,θ,φ ) is a well behaved periodic\nfunction that defines a family of transformations 18. The\nperiodicity of ω preserves the toroidal and poloidal character of the two angular coordinates.\nStarting with some given straight-field line coordinate\nsystem, we want to understand how to construct a set\nwith a Jacobian of the GBC form. Thus, we need to\nknow how to transform the coordinate Jacobian induced\nby ω. This reads\nJ′−1 = ∇ψ×∇(θ−ιω) ·∇(φ−ω).\nHere {ψ,θ′,φ′}represents the newly defined straight field\nline coordinates whose associated Jacobian is J′. This\nequation may be recast into the form of a magnetic differential equation,\nB ·∇ω= 1\nJ − 1\nJ′. (4)\nNow, let us require19\nJ′= Bα(ψ)\nB2 . (5)\nIn order for the magnetic differential equation Eq. (4)\nto have a single-valued solution for the transformation\nfunction ω, Eq. (4) must satisfy Newcomb’s criterion 20.\nAccording to Newcomb, for a magnetic differential equation B ·∇f = s to have a single-valued solution for f,\nthe source term smust satisfy the line-integral condition\nalong a field line,\n∮\nsdl\nB = 0. (6)\nFor our problem s= 1/J− 1/J′. Start with,\n∮ 1\nJ\ndl\nB =\n∮\nB ·∇φdl\nB =\n∮\ndφ= 2πn.\nHere we have used the definition of the Jacobian in terms\nof the magnetic fieldB·∇φ= ∇ψ×∇θ·∇φ= 1/J, where\nφ and θ are, respectively, toroidal and poloidal angular\ncoordinates that increase by 2π in going around the long\nor short toroidal circular paths. We also considered the\nrotational transform ι = n/m, where n,m ∈Z and are\ncoprime.\nFIG. 1. Ribbon surface. Ribbon defined by two closely\nlying rational magnetic field lines labelled by C1 and C2.\nNow, consider Newcomb’s condition on the last term\non the right-hand-side of Eq. (4), and define the closed\nline integral Iso that,\n∮ 1\nJ′\ndl\nB = I\nBα(ψ). (7)\nFor Newcomb’s condition on Eq. (4) to hold, the following\nmust be true:\nBα(ψ) = I\n2πn. (8)\nWith this choice\n∮\nB ·∇ωdl\nB = 0. (9)\nThen there must exist a single-valued solution ωthat enacts the coordinate transformation into the set associated\nwith the Jacobian (5).\nFor Eq. (8) to hold, it is necessary for Ito be a flux\nfunction. This condition may be written in the form,\nI(ψ) =\n∮\nB ·dr, (10)\nalong a magnetic field line. Focus on the case of rational\nfield lines, for which the condition is most stringent. To\nproceed further, consider a non-self-intersecting ribbon\nover a flux surface bounded by two adjacent magnetic\nfield lines (see Fig. 1). Denote the line integrals along\neach of these lines by C1 and C2. We may then write,\nusing Stoke’s theorem,\n∮\nC1\nB ·dr −\n∮\nC2\nB ·dr =\n∫\nrib\nj ·dS, (11)\nwhere the surface element is taken to be perpendicular\nto the flux surface. The integral over the surface may be\nwritten as,18\n∫\nrib\nj ·dS =\n∫ α0+δα\nα0\ndα\n∮ dl\nBj ·∇ψ.\nHere α labels the field lines on the surface. Now, if Iis\nto be truly a flux function, then following (11) the last\nRodr´ ıguez et al. 4\nsurface integral must vanish. And it must do so for all\nfield line labels α0 and adjacent field lines δα. This gives\nthe necessary and sufficient condition,\n∮ dl\nBj ·∇ψ= 0. (12)\nThis subclass of magnetic fields grant the required form\nof I, and thus the single-valued solution to Eq. (4). This\nsubclass includes the stronger constraint j ·∇ψ= 0 (for\nwhich the coordinates reduce to Boozer coordinates), or\nmore to our concern, the QS scenario (see Appendix A).\nNote that magnetic fields that possess nested surfaces\nhave the property ⟨j ·∇ψ⟩= 0, following an application\nof ∇·(B ×∇ψ). However, the Newcomb condition (12)\nis more stringent.20\nRestricting ourselves to this subclass of fields, we can\nalways choose Bα as in (8). This choice might appear\nartificial, especially given the presence of the discrete n.\nHowever, we may relate this to the surface average over\nirrational flux surfaces. To see this, we write,\nBα = 1\n2πn\n∮\nB2 dl\nB = 1\n4π2n\n∫ 2π\n\ndα\n∮\nB2 dl\nB \nn turns\n=\n= 1\n4π2\n∫ 2π\n\ndα\n∮\nB2 dl\nB \n1 turn\n= V′\n4π2 ⟨B2⟩, (13)\nwhere V′ =\n∫2π\n0 dα\n∫\ndl/B = V′⟨1⟩is the usual volume\nψ derivative. Bα is, therefore, a well behaved quantity\nfor both rational and irrational surfaces.\nAs it stands, with an appropriate choice of Bα, and\nrestricting ourselves to the subclass of fields satisfying\nEq. (12), one may perform a coordinate transformation\nthat provides a Jacobian of the desired form (5). It remains to be shown that this new coordinate system is\nwell-behaved. By this we mean that the Jacobian does\nnot vanish nor diverge. To see this, it is most useful to\nrewrite J′in terms of (13),\nJ′= Bα\nB2 = V′\n4π2\n⟨B2⟩\nB2 . (14)\nIt is clear that J′has a definite sign, as given by the sign\nof V′. Thus, the Jacobian will never vanish nor diverge,\ngiven that B2 >0.\nC. Magnetic field in generalized Boozer coordinates\nWe have shown that under the assumption that the\nmagnetic field is quasisymmetric, we have a straightfield-line coordinate system, GBC, with a Jacobian J =\nBα(ψ)/B2. In this coordinate system and following\nSec. II A, a quasisymmetric field is one whose magnitude\ncan be expressed in GBC as B = B(ψ,θ −˜αφ). This\nform is analogous to the Boozer formulation of QS but\nis only subordinated to the configuration being QS and\nnot j ·∇ψ= 0.\nBefore proceeding to analyze some of the properties of\nQS and other governing equations in GBC, we explicitly\nwrite the covariant and contravariant forms for B. The\ncovariant form is\nB = Bθ∇θ+ (Bα −ιBθ)∇φ+ Bψ∇ψ. (15)\nThe usual covariant function Bφ has been deprecated for\nBα. This is the flux function that appears in the Jacobian\n(5). The simplicity of (15) shows why it was convenient\nto choose the Jacobian of GBC to have the particular\nform in (5). To obtain (15), it is sufficient to take its\nscalar product with the contravariant representation,\nB = ∇ψ×∇θ+ ι∇φ×∇ψ, (16)\nand capitalize on the definition of J. Compared to\nBoozer coordinates, the covariant function Bθ in GBC is\nnot necessarily a flux function. Thus, GBC is an extension of Boozer coordinates. When j ·∇ψ = 0, however,\nand as previously pointed, GBCs reduce to Boozer coordinates. So far, the forms in (15) and (16) have only\nrequired of the existence of GBC, and not of QS per se\n–other than to guarantee (12). To enforce the latter, one\nneeds to specify |B|as a symmetric function. 14,16,21\nIII. DESCRIBING WEAK QUASISYMMETRY IN GBC\nHaving developed GBC, let us see what this coordinate\nsystem can teach us about weak QS. We first write down\nthe complete set of equations that describe a weakly quasisymmetric magnetic field. The first relevant equation\nequates the covariant (15) and contravariant (16) forms\nof the magnetic field. The equation reads,\nBθ∇θ+ (Bα −ιBθ)∇φ+ Bψ∇ψ=\n= ∇ψ×∇θ+ ι∇φ×∇ψ. (17)\nTo specify that we are using GBC, and to introduce the\nquasisymmetric condition, we require\n∇ψ×∇θ·∇φ= Bα(ψ)\nB2(ψ,θ −˜αφ). (18)\nThis set of equations (17) and (18) is entirely selfcontained. It describes a general magnetic field (a vector field satisfying ∇· B = 0 = B ·∇ψ, without a particular form of equilibrium) that is quasisymmetric —\nno more, no less. Such equations have been recently\nused in near-axis expansions with anisotropic pressure\nequilibria.14,16 Equations (17) and (18), referred to as\nthe co(ntra)variant and Jacobian equations respectively,\nwere there expanded systematically. A more thorough\nand complete exploration of the expansion of the magnetic equations and its implications will be presented in\na separate publication.\nRodr´ ıguez et al. 5\nEquations (17) and (18) apply beyond near-axis expansions. Alternatively, one could study the behavior\nof this system in a perturbative sense but around a surface rather than the axis. 21 This could shed some light\nto standard optimization approaches to QS 9,10.\nBeyond the set of equations (17) and (18), the formulation of weak QS in terms of GBC can be used to compare\nweak QS to other (quasi)symmetric forms. This comparison to strong and axisymmetry will help frame the notion\nof weak QS in the larger space of configurations.\nA. Comparison to strong quasisymmetry\nStrong QS is a more resrictive form of QS compared to\nits weak form3,12. Weak QS is the necessary and sufficient\ncondition for the leading guiding centre dynamics to have\nan approximately conserved momentum to leading gyrocentre order.3 This condition can be written as in (1),\nbut is most naturally given as,\nu ·∇B = 0, (19)\nB ×u = ∇Φ(ψ), (20)\n∇·u = 0. (21)\nThe vector fieldu is defined by these equations and points\nin the direction of symmetry. In strong QS the conserved momentum for the particle dynamics is exact for\nthe first-order guiding centre Lagrangian.3,5,12 In the notation that explicitly introduces the symmetry vector u,\nstrong QS is equivalent to weak QS – i.e., Eqs. (19)-(21)–\nplus the constraint\nj ×u + ∇C = 0, (22)\nwhere C = B·u. Note that only the b component of this\nadditional constraint is contained in the weak formulation of the problem. We now explore the significance of\n(22) in the context of GBC.\nTo begin, we need to construct u. From Eqs. (19)(21),3\nu = ¯ι∇ψ×∇χ\nB ·∇χ , (23)\nwhere χ= θ−˜αφ, and it is convenient to use χas part of\nthe coordinate triplet {ψ,χ,φ }. We have made the choice\nΦ′= ¯ι so that u ·∇φ = 1, for simplicity. Scaling of the\nflux label for u leaves the weak QS conditions unchanged\n(see Appendix B for further discussion on the gauge and\nthe particular choice). This form of u is equivalent to\nEqs. (19)-(21) only if we enforce B = B(ψ,χ) and the\ncoorrdinate system B ·∇χ= ¯ιJ−1 with the Jacobian in\n(5). The parameter ¯ι= ι−˜α has been defined.\nFrom (15) and (23), we find,\nC = Bα −¯ιBθ. (24)\nThis relation (24), together with (23) and the definition\nof C can be used to write the gauge-independent form,\nB ·∇ψ×∇B = Bα −¯ιBθ\n¯ι B ·∇B. (25)\nThe magnetohydrostatic form of this equation has been\nused previously18,22.\nThe contravariant form of u together with (16) give\nsimple expressions for directional derivatives in GBC.\nThe differential operators can be simply written as partial derivatives,\nB ·∇ = J−1 (¯ι∂χ + ∂φ) , u ·∇ = ∂φ. (26)\nSince the Jacobian is quasisymmetric from (5), the two\noperators B·∇and u·∇commute with each other11. This\ncommutation property is made manifest in the GBC.\nThe symmetry field u can also be written in the following covariant form:\nu = uψ∇ψ+ uχ∇χ+ uφ∇φ. (27)\nTaking the scalar product with u and B we obtain\nuφ = u2, ¯ιuχ + uφ = C\nJ−1 . (28)\nTo complete the family of vectors required for the\nstrong quasisymmetric condition (22), we need a closed\nform for j in GBC. From the curl of the covariant form\nof B in Eq. (15), we obtain\nj = ∇Bψ×∇ψ+ ∇Bθ×∇χ+ ∇(Bα−¯ιBθ) ×∇φ. (29)\nUsing (29) and (24), we can show that\nj×u+∇C = (u·∇Bψ)∇ψ+ (u·∇Bθ) (∇χ−¯ι∇φ) .\n(30)\nThe simplicity of (30) is due to the choice of GBC.\nRecall that strong QS requires the expression j ×u +\n∇C to be identically zero. This means that all the terms\non the right side of (30) need to vanish. That is to say, the\ncovariant functions (Bψ,Bθ) are required to be quasisymmetric. If Bθ is quasisymmetric, then C is automatically\nso from (24). In an explicit coordinate representation,\nusing (26), we may write Bθ(ψ,χ) and Bψ(ψ,χ).\nThus, the GBC representation provides an elegant way\nto formulate strong QS, which can now be understood as\nweak quasisymmetry plus the conditions that Bψ and Bθ\nare QS. In other words, not only is B QS but so are Bθ\nand Bψ.\na. Implications for near-axis expansion . We refer\nto [14] and [16] for a detailed treatment using near-axis\nexpansions of the weakly quasisymmetric problem. The\nprocedure is based on expanding all governing equations\ndescribing the weak quasisymmetric field in some form of\nequilibrium in powers of the distance from the magnetic\naxis. To do so efficiently, both the field and equations\n–including Eqs. (17) and (18)– are expressed in GBC. It\nwas shown there that when equilibria with anisotropic\npressure are considered, the common overdetermination\nproblem7,8 that limits the expansion is overcome. The\nnumber of governing equations becomes the same as that\nof functions to be solved.\nRodr´ ıguez et al. 6\nExtending the expansion to strong QS is straightforward. From the discussion above, the only difference\nis that the covariant functions Bθ and Bψ are both\nquasisymmetric rather than general functions of space.\nIn practical terms, this simply leads to more restricted\nTaylor-Fourier expansions of those functions; the coefficients that were functions of φ become constants. This\nrestriction in freedom once again leads us back to the\nGarren-Boozer overdetermination problem. In fact, it\ndoes so the same way as it did in the case of MHS equilibrium. The restriction on the covariant functions imposes\nvery severe constraints on the allowed geometry. The\nonly way to escape this impasse is to assume axisymmetry (φ independent). Once again, consistent with what\nwe observed in [14], the asymmetry of the covariant representation of B appears to be vital to the construction\nof QS solutions.\nB. Comparison to axisymmetry\nWe have seen that the strong formulation of QS is more\nconstraining than its weak form. We would also like to\ncompare QS to the limiting case of axisymmetry. We\nshall think of this case as a symmetry generated by rotation in space: the system is invariant under rotations\nabout an axis. In Euclidean space, a rotation is an isometry, and it is generated by a vector field known as a\nKilling vector. Using the notion of a Killing vector, we\nwant to explore how ‘far’ the weak concept of QS is from\nthis ‘true’ symmetry.\nA measure of the departure of a symmetry generator from a Killing vector is the so-called deformation\nmetric.12 Taking u to represent the symmetry vector for\nQS, the idea is to see how far it is from being a Killing\nvector. A vector field v is Killing if and only if the deformation tensor Lvg= 0. Here Lu denotes the Lie derivative along u and g is the Euclidean metric. In 3D, this\nmay be written as,\nLug= ∇u + (∇u)T. (31)\nEvaluating this tensor for a quasisymmetric configuration\nshould then provide information regarding the closeness\nto an isometry. It is convenient12 to evaluate this rank-2\ntensor in a basis defined by {B,u,∇ψ}, a triplet that we\nshall take to be independent. Then,\n[\n∇u + (∇u)T]\n·B =j ×u + ∇C, (32)\n[\n∇u + (∇u)T]\n·u =∇u2 −u ×∇× u, (33)\n[\n∇u + (∇u)T]\n·∇ψ=∇ψ×∇× u + 2∇ψ·∇u, (34)\nwhere the weak quasisymmetric properties have been\nused where necessary. Equation (33) is what Burby et al.\ncall w.11 We shall explore this vector w in more detail\nafter obtaining an explicit form for Lug. Using (32)-(34),\nand projecting once again onto the non-orthogonal basis\ntriplet, we obtain\nLug=\n\n\n0 u ·∇C ∇ψ·(j ×u + ∇C)\n... u ·w w ·∇ψ\n... ... −u ·∇|∇ψ|2\n\n. (35)\nThe matrix is symmetric by construction. The content\nof its elements can be made clearer using GBC explicitly.\nThe top row, corresponding to (32), has already been\ndealt with, as it is precisely the piece corresponding to\nstrong QS. We made the observation that for this condition to be satisfied, (30) required u·∇Bθ = 0 = u·∇Bψ.\nFor the other components, the vector field w = ωu ×\nu + ∇u2, ω u = ∇×u is key. Using the covariant form\nof u we obtain the curl of the vector u in the form\nωu = ∇uψ ×∇ψ+ ∇uχ ×∇χ+ ∇uφ ×∇φ. (36)\nTaking the cross product with u, using the orthogonality\nof u with ∇ψ and ∇χ, and (24) and (28) we get\nw = (u ·∇uψ)∇ψ+ (u ·∇u2)\n(\n∇φ−1\n¯ι∇χ\n)\n+\n−J(u ·∇Bθ)∇χ, (37)\nwhich implies that\nB ·w = −¯ιu ·∇Bθ, (38)\nu ·w =u ·∇u2. (39)\nMost importantly, a vanishing w implies that the covariant components of the symmetry vector as well as Bθ are\nquasisymmetric.\nTo complete the simplification of the metric tensor, we\ninvoke B2 = ( C2 + |∇ψ|2)/u2, which follows from the\ndefinition of u. This means,\nu ·∇|∇ψ|2 = B2u ·∇u2 −u ·∇C2. (40)\nWith this coordinate representation, the dependence of\nthe various metric pieces is made explicit. We may then\nschematically present the dependence of Lug as follows,\nLug∼\n\n\n0 ∂φBθ ∂φBθ, ∂φBψ\n... ∂ φu2 ∂φu2, ∂φuψ\n∂φBθ\n... ... ∂φBθ, ∂φu2\n\n. (41)\nThe boxed expressions are meant to indicate that the\ncorresponding tensor component vanishes if the expressions there do. If the tensor (41) is to vanish, then the\nsymmetry vector would correspond to a rotation. This is\nnot surprising if one looks at what it means for the components in (41) to vanish. Axisymmetry is reached when\nthe covariant components of the magnetic field and the\nsymmetry vector are themselves symmetric. The latter is\nintimately related to the geometry, as we may see when\nwriting u ∝∂φx|χ,ψ.\nRodr´ ıguez et al. 7\nFrom (41), it follows that in some sense, weak QS is\nfar from being an isometry. This is so because only one\nof the components of the tensor exactly vanishes. The\nφ dependence of the functions Bψ, Bθ, u2, and uψ takes\nthe configuration away from axisymmetry. These apparent four degrees of freedom (especially those involving\nu) may not be independent and involve highly non-linear\ncombinations—they should ultimately be related through\nthe quasisymmetric magnetic equations. Anyhow, the\nfield-line dependence is key in distinguishing the weakly\nquasisymmetric form from, say, an axisymmetric tokamak.\nTo make a comparative measurement of the departure\nfrom axisymmetry, consider now the case of strong QS.\nIn this case, following (22), the first whole row (and thus\nalso column) of (41) drop. The remaining dependence\nalso simplifies, and the system is precluded from being\nan isometry, a priori, through the φ dependence of u2\nand uψ only, which is consistent with the work by Burby\net al.12\nImposing additional properties to the field may also\naffect the form of the deformation tensor. An example\nwould be a particular form of force balance. We now\nexplore how the magnetics and equilibria are linked.\nIV. QUASISYMMETRY AND EQUILIBRIA\nLet us consider the force balance part of the problem.\nGenerally, a magnetic equilibrium with some arbitrary\nforce F reads,\nj ×B = F. (42)\nAs we argued in Sec. II, we are concerned in this work\nwith a general fluid force F. Its connection to the microphysics is not considered. Let us express the left-hand\nside of (42) in GBC. Using the contravariant form for\nthe current (29) together with (24), we obtain\nj ×B =\n[\nB ·∇Bψ −J−1 (B′\nα −¯ι′Bθ)\n]\n∇ψ+\n+ (B ·∇Bθ) (∇χ−¯ι∇φ) , (43)\nwhich is an explicit coordinate representation of j ×B.\nThe form of (43) mirrors the form of (30). In this case,\nthe magnetic differential operators substitute the directional derivatives along u. We note that (43) does not\nhave any component along B, as can be checked by taking the dot product with B.\nThe form of (43) puts constraints on the allowable\nforms for F. As already noted, F ·B = 0 must hold\ntrue. Otherwise the system would be imbalanced, as the\nmagnetic field is unable to exerts forces alongB. Because\nof this reduction in the dimensionality of F and in view\nof (43), it is convenient to write\nF = Fψ∇ψ+ Fα(∇χ−¯ι∇φ) . (44)\nAn alternative form would be to use the contravariant\nform of B (16) in (42) to get\nF = (J ·∇χ−¯ιJ ·∇φ) ∇ψ−(J ·∇ψ) (∇χ−¯ι∇φ) .\n(45)\nSubstituting (43) and (44) into (42) we get two magnetic\ndifferential equations\nB ·∇Bψ = Fψ + J−1 (B′\nα −¯ι′Bθ) , (46a)\nB ·∇Bθ = Fα = −J ·∇ψ. (46b)\nTherefore, the generalized force-balance condition is\nequivalent to two magnetic differential equations (MDEs)\nand B ·F = 0. If solutions to these equations can be\nfound together with the magnetic equations (17) and\n(18), we will have obtained a quasisymmetric configuration in equilibrium.\nLet us describe in more detail the implications of these\nequations. First look at the simpler (46b). This equation\nhas two pieces to it. First, and regardless of the assumed\nform for Fα, it follows from weak QS that B ·∇Bθ =\n−j ·∇ψ (see Appendix A as well). This imposes the\ncondition (II B) on the field. Secondly, the component\nFα of the forcing F directly sets the off-surface current.\nThis means that,20\n∮ dℓ\nBFα = 0. (47)\nWe may not choose Fα arbitrarily. It must satisfy (47) if\nthe force is to be consistent with QS –condition like (12).\nThen the magnetic differential equation can be satisfied,\nand one can directly relate Bθ and Fα up to a flux function. In Fourier representation, it is clear that the φcontent of Bθ will be non-zero only if that of Fα is (and vice\nversa). In the light of (41), choosing ∂φ(j·∇ψ) = 0 brings\nthe quasisymmetric configuration closer to an isometry.\nThis freedom in the form of Bθ does not exist in the\nstrong formulation of QS, for which j·∇ψis independent\nof the field line label.\nA similar analysis is suitable for (46a). The appropriate Newcomb condition in this case is,\n∮ dℓ\nB\n[\nFψ + J−1 (B′\nα −¯ι′Bθ)\n]\n= 0. (48)\nThis condition may be understood as an averaged radial\nequilibrium equation. A similar solvability condition can\nbe found, for the special case of MHS, in [4], where the\nnotion of a QS field is presented as one that satisfies the\nNewcomb conditions. Given Eq. (48), Eq. (46a) relates\nBψ, Bθ (or Fα) and Fψ. Once again, we see the close\nrelationship between the forcing, the magnetic covariant\nrepresentation, and the deviation from axisymmetry. Aφ\ndependence on Bψ will bring a finite deviation of u from\nbeing a Killing vector. However, it will also force Fα\nand Fψ to have a φ dependence which may require very\nparticular shaping of the forces. This observation is consistent with the Constantin-Drivas-Ginsberg theorem 23,\nRodr´ ıguez et al. 8\nin which the forcing is seen to be intimately related to the\ndeviation from an isometry. Here the asymmetric geometry, quasisymmetry, and the forcing are all intimately\nconnected.\nWhen the magnetic differential equations imposing\nforce balance are brought together with the magnetic\nequations from the previous section, it is not obvious\nhow the system of equations is to be interpreted: what\nis to be taken as an input and what should be solved for.\nJust as an analogy, in the Grad-Shafranov equation, it is\nclear that p and F are inputs, and ψ is the output. In\nthe present problem, we have the construction of GBC\nin addition to the various magnetic covariant forms and\nthe components of F. Motivated by the treatment in [3]\n(which deals with a special case of the above), we propose as a possibility for a well-posed problem (46b) to be\nsolved for Bθ given Fα, while Fψ is the output of (46a),\nwith the function Bψ specified from the magnetic equations. It is not a trivial matter to determine a convenient\nway in which to formulate the problem. A more elaborate discussion on this procedure and its implications on\nconstructing solutions is left to future work.\nA. Ideal MHD: j ×B = ∇p(ψ)\nLet us now revisit the limit of ideal MHD without\nflows, j ×B = ∇p. More general forms will be discussed\nelsewhere, together with a more systematic treatment of\nthe quasisymmetric system of equations.\nIn ideal MHD, from j ·∇p(ψ) = 0 it follows that\nB ·∇Bθ = 0. (49)\nThus taking Fα = 0 forces Bθ into a flux function. Of\ncourse, this also means that u·∇Bθ = 0. Furthermore, as\nFψ = p′, in static ideal MHD, (43) leads to the magnetic\ndifferential equation for Bψ,\nB ·∇Bψ = p′(ψ) + J−1 (B′\nα −¯ι′Bθ) (50)\nSince Bψ must be a single-valued function the fluxsurface average of (50) gives\np′(ψ) + ⟨B2⟩\nBα\n(B′\nα −¯ι′Bθ) = 0. (51)\nIf we choose the forms ofB, p, and ι, this pins the form of\nBα down. Now, looking back to (50), every term on the\nright-hand side is quasisymmetric. Therefore, Bψ must\nalso be quasisymmetric if it is to satisfy the force-balance\nequation. Note that we have already recognized this constraining requirement on the form of Bψ as the origin\nof the Garren-Boozer overdetermination problem14. The\nNewcomb condition on this equation can be recognized\nas the condition to avoid Pfirsch-Schl¨ uter current singularities.\nThe simplifications due to ideal static MHD leads to\nthe vanishing of (30). Therefore, in this limit weak QS\nis identical to strong QS.\nOne can further show using (29), (15), (23) and (50),\nj = −1\n¯ι∂ψ(Bα −¯ιBθ)B −1\n¯ιp′(ψ)u, (52)\nwhere the gauge choice for u has been made Φ′= ¯ι. The\nexpression for j ought to be u-gauge independent, as it is\na physical quantity. The B piece as written is gauge independent, but the u term is not. The ¯ιfactor in the latter\nis to be interpreted as Φ ′. This equation had been obtained previously 11,12 using coordinate free, differential\nforms (see Appendix C). Two special cases of ideal MHD,\na) vacuum (j = 0) and b) force-free ( j = λ(ψ,α)B), are\nworth pointing to for their importance in plasma physics.\nFor both these cases p′(ψ) = 0. From (52) we see that for\nthe magnetic field to be curl-free (vacuum) and quasisymmetric C′(ψ) = 0; i.e., C(ψ) must be a constant. For quasisymmetric force-free fields, we must have λ= −C′(ψ).\nNote that in strong QS these conclusions follow directly\nfrom the equation j ×u+∇C = 0 with j = 0 and j = λB.\nV. CONCLUSIONS\nIn this paper, we have presented, defined, and discussed a straight field line coordinate system which is\nnatural for the representation of general-equilibria quasisymmetric magnetic fields: generalized Boozer coordinates. We proved the existence of the said coordinate system for the subset of fields for which\n∮\nj·∇ψdl/B = 0, to\nwhich quasisymmetric fields belong. These coordinates\nreduce to Boozer coordinates when j ·∇ψ= 0.\nThe explicit form of the symmetry in this coordinate\nrepresentation enables a simple formulation of the quasisymmetric problem. We explicitly construct the governing equations setting clearly the foundation for future investigations, including expansion 14,16 and global\napproaches. Exploiting GBC, we explicitly show the essential differences between the weak and strong formulations of QS and between quasisymmetry and axisymmetry. Weak QS generally lies far from axisymmetry, for\nwhich many functions describing the field and symmetry\nneed to be symmetric.\nWe also included a set of simple magnetic differential\nequations that fully describe equilibrium with an arbitrary macroscopic force to complete the treatment. The\nproperty of QS, together with the force-balance structure, imposes requirements on the forcing terms in the\nform of Newcomb conditions. In addition, the equations\nestablish clear connections between QS, forcing, and departures from axisymmetry.\nACKNOWLEDGEMENTS\nWe are grateful to J. Burby, N. Duigan, J. Meiss,\nand D. Ginsburg for stimulating discussions. This research is supported by grants from the Simons FoundaRodr´ ıguez et al. 9\ntion/SFARI (560651, AB) and DoE Contract No DEAC02-09CH11466.\nDATA A VAILABILITY\nData sharing is not applicable to this article as no new\ndata were created or analyzed in this study.\nAppendix A: Off-surface current and QS\nIn this appendix we directly show how the triple vector\nformulation of QS in Eq. (1) imposes a constraint on the\noff-normal component of the current at every magnetic\nsurface.\nLet us start by noting that B ·∇B = f(ψ,B) is a\nconsequence of (1). In that sense, we can reshape the\ntriple vector condition in the convenient form,\n∇ψ×∇B·∇\n( B2\nB ·∇B\n)\n= 0. (A1)\n(For now we shall not worry about B·∇B = 0.) We may\nexpress this as a divergence as ∇·(∇ψ×∇B) = 0. Separating the argument of the divergence into a component\nparallel to the magnetic field and perpendicular to it, we\nmay rewrite (A1),\n∇·\n[∇ψ×∇B·B\nB ·∇B B + ∇ψ×B\n]\n= 0. (A2)\nIt is customary to define C = ∇ψ×∇B·B/(B ·∇B).\nThis can therefore be rewritten, using ∇·B = 0,\nB ·∇C = j ·∇ψ. (A3)\nIt is then clear that the Newcomb condition 20,\n∮\nj ·∇ψdl\nB = 0, (A4)\nwith the line integral taken along closed magnetic field\nlines. For irrational field lines, this amounts to ⟨j·∇ψ⟩=\n0, which is true of all magnetic fields with flux surfaces\nB ·∇ψ = 0. The condition (A4) is a more constraining\ncondition than the flux surface average one 20.\nThe division byB·∇Bseems to be ill-defined at the extrema of the magnetic field along the field lines. However,\nthe above holds, first, arbitrarily close to these extrema,\nand thus would expect to hold by continuity. The fact\nthat the condition holds can be seen by looking at the\nfundamental formulation of QS. 3 As for the behavior of\nC, this is a physical quantity related to the width of banana orbits of bouncing particles. If deeply trapped and\nbarely trapped particles are to be kept, then physically\nC must be finite at the extrema. Once we express everything in terms of GBC, the apparent divergence of C\ndisappears as both the numerator and denominator are\nproportional to ∂χB = 0 when B ·∇B = 0.\nAppendix B: Gauge freedom in weak u\nAs noted, the definition of u by the weak quasisymmetry equations (19)-(21) is invariant to the relabelling of\nΦ. A rescaling of the flux surface label together with u\nleaves the equations invariant, and should therefore have\nno physical implication in the description of quasisymmetry. Keeping the general gauge label Φ( ψ) of (20),\nEq. (23) reads,\nu = ∇Φ ×∇χ\nB ·∇χ . (B1)\nThen it folllows,\nC = Φ′\n¯ι (Bα −¯ιBθ). (B2)\nand\nu ·∇ = Φ′\n¯ι ∂φ. (B3)\nIn Sec. III A we then went on to evaluate how the symmetry vector u as defined by weak QS compares to strong\nQS. To do so, we needed to evaluate (30), the third of\nthe strong QS conditions. The first thing to note is\nthat this equation is not gauge-invariant. This is, of\ncourse, an added complication to the problem. In a sense,\nthis gauge-symmetry breaking leaves no unique form for\nj ×u + ∇C, but rather a whole family.\nThis can be seen explicitly if instead of restricting ourselves to the special case Φ ′ = ¯ι, we keep Φ general.\nThen, (30) can be written in the form,\nj ×u + ∇C =\n[\nu ·∇Bψ −CΦ′\n¯ι ∂ψ\n( ¯ι\nΦ′\n)]\n∇ψ+\n+ (u ·∇Bθ) (∇χ−¯ι∇φ) . (B4)\nSo the question may be reformulated: what are the conditions for this expression to vanish? As before one piece\nis u ·∇Bθ = 0. Now, the other component has the form\nof a differential equation along the direction of symmetry.\nWe may write it explicitly as,\n∂φBψ −(Bα −¯ιBθ)∂ψln(¯ι/Φ′) = 0. (B5)\nFor the equation to be consistent it must then hold that\neither ¯ι= aΦ′where ais a constant, or\n∫\n(Bα−¯ιBθ)dφ=\n0. The latter is a condition on the physically relevant\npart of the problem. Thus, it is minimal to choose Φ′= ¯ι\nto satisfy this condition. Then we are simply left with\nu ·∇Bψ = 0, which is the choice made in the main text.\nAppendix C: Coordinate independent form of\nquasisymmetric current\nIn this appendix we present a brief derivation of (52)\nwithout using coordinates. Start from,\nB ×u = ∇Φ (C1)\nRodr´ ıguez et al. 10\nwith Φ some single-valued function. With magnetic flux\nsurfaces labelled by Φ, to which B as well as the vector\nfield u are tangent. From this condition, one may also\nwrite the symmetry vector field in the following form,\nu = 1\n|B|2 (CB −B ×∇Φ). (C2)\nHere C = B ·u is some arbitrary function of space, as\nthe constraint equation only describes the perpendicular\npart of u.\nBecause of magnetostatic equilibrium, we hav\nj ×B = ∇p (C3)\nwhere j = ∇×B is the plasma current density. Because\nB ·∇p = 0, it follows that p = p(ψ), again assuming\nthat B is irrational. By continuity, we argue that this\nmust be true for the rational cases as well (provided the\nrotational transform is not constant ι′̸= 0).\nBecause the plasma pressure is also a flux function, the\ncurrent densityj also lives on magnetic flux surfaces. Furthermore, requiring ∇p to be nowhere vanishing (except\nthe axis), the magnetic field and the currents are guaranteed to be non-collinear. Therefore, at every point on\na flux surface, we may define a tangent plane spanned by\nj and B.\nBecause u also exists in this subspace, we can express\nit in the chosen basis as,\nu = aj + ˜βB\nTaking the cross product of this form of u with B, and\nmaking use of (C1) and (C3), it is clear that a= a(ψ) =\n−Φ′/p′, where the prime denotes a derivative with respect to the flux ψ.\nLet us now apply (21), namely ∇·u = 0, on our form\nof u,\n∇·u = B ·∇˜β = 0 ∴ ˜β = ˜β(ψ)\nso that,\nj = −p′(ψ)\nΦ′ u + β(ψ)B. (C4)\nUsing the form (C2), we can then rewrite the expression\nfor j in the form,\nj =\n(\nβ(ψ) −p′(ψ)C\nB2Φ′\n)\nB + p′(ψ)\nB2 B ×∇ψ (C5)\nThe gauge-independent combination C/Φ′= (B ·∇ψ×\n∇B)/(B ·∇B) here is the same as appears in (25), and\nit is a flux function. There we found a closed form for\nthis combination in terms of the covariant representation\nof B in GBC. Equation (C5), together with ∇· B = 0\nand B ·∇ψ = 0 constitute the whole set of governing\nequations for the QS, MHS field. The problem in this\ncase can be formulated by providing β, C/Φ′ and p as\ninput flux functions and solving for B.\nAssuming that we have strong QS, β = −C′/Φ′, obtaining an expression equivalent to that obtained in (52).\nThe assumption of strong QS was, however, not necessary when working with the coordinate representation.\nThe adoption of some form of covariant representation\nof B seems to be necessary to obtain such a relation.\nTo that end, GBC is convenient, though not necessary.\nIn fact, [11] employs the B♭ form (the generalisation of\nthe covariant representation of the magnetic field), without specifying coordinates, to prove (52). Note, however,\nthat the difference in the form of β is merely a matter of\nhow the flux function is defined. The problem has β or\nΦ′as free flux function inputs.\n1J. N¨ uhrenberg and R. Zille, Physics Letters A129, 113 (1988).\n2A. H. Boozer, The Physics of Fluids 24, 1999 (1981).\n3E. Rodr´ ıguez, P. Helander, and A. Bhattacharjee, Physics of\nPlasmas 27, 062501 (2020).\n4M. Tessarotto, J. L. Johnson, and L. J. Zheng, Physics of Plasmas 2, 4499 (1995).\n5M. Tessarotto, J. L. Johnson, R. B. White, and L. Zheng, Physics\nof Plasmas 3, 2653 (1996).\n6A. H. Boozer, Physics of Fluids 26, 496 (1983).\n7D. A. Garren and A. H. Boozer, Physics of Fluids B: Plasma\nPhysics 3, 2822 (1991).\n8M. Landreman and W. Sengupta, Journal of Plasma Physics 84,\n905840616 (2018).\n9A. Bader, M. Drevlak, D. T. Anderson, B. J. Faber, C. C. Hegna,\nK. M. Likin, J. C. Schmitt, and J. N. Talmadge, Journal of\nPlasma Physics 85, 905850508 (2019).\n10S. Henneberg, M. Drevlak, C. N ˜A¼hrenberg, C. Beidler,\nY. Turkin, J. Loizu, and P. Helander, Nuclear Fusion 59, 026014\n(2019).\n11J. W. Burby, N. Kallinikos, and R. S. MacKay, Journal of Physics\nA: Mathematical and Theoretical 54, 125202 (2021).\n12J. W. Burby, N. Kallinikos, and R. S. MacKay, Journal of Mathematical Physics 61, 093503 (2020).\n13J. W. Burby and H. Qin, Physics of Plasmas 20, 012511 (2013).\n14E. Rodr´ ıguez and A. Bhattacharjee, Physics of Plasmas 28,\n012508 (2021).\n15D. A. Garren and A. H. Boozer, Physics of Fluids B: Plasma\nPhysics 3, 2805 (1991).\n16E. Rodr´ ıguez and A. Bhattacharjee, Physics of Plasmas 28,\n012509 (2021).\n17M. D. Kruskal and R. M. Kulsrud, The Physics of Fluids 1, 265\n(1958).\n18P. Helander, Reports on Progress in Physics 77, 087001 (2014).\n19A more general form J ′ = J ′(ψ,B) could have been demanded.\nHowever, one may show in that case that the system may always\nbe cast into the form here employed.\n20W. A. Newcomb, The Physics of Fluids 2, 362 (1959).\n21W. Sengupta, E. J. Paul, H. Weitzner, and A. Bhattacharjee,\nJournal of Plasma Physics 87, 905870205 (2021).\n22E. J. Paul, T. Antonsen, M. Landreman, and W. A. Cooper,\nJournal of Plasma Physics 86, 905860103 (2020).\n23P. Constantin, T. D. Drivas, and D. Ginsberg, Journal of Plasma\nPhysics 87, 905870111 (2021).",
"[ABSTRACT]\nReal-world time-series datasets are often multivariate with complex dynamics. To capture\nthis complexity, high capacity architectures like recurrent- or attention-based sequential\ndeep learning models have become popular. However, recent work demonstrates that simple\nunivariate linear models can outperform such deep learning models on several commonly\nused academic benchmarks. Extending them, in this paper, we investigate the capabilities of\nlinear models for time-series forecasting and present Time-Series Mixer (TSMixer), a novel\narchitecture designed by stacking multi-layer perceptrons (MLPs). TSMixer is based on\nmixingoperationsalongboththetimeandfeaturedimensionstoextractinformationefficiently.\nOn popular academic benchmarks, the simple-to-implement TSMixer is comparable to\nspecialized state-of-the-art models that leverage the inductive biases of specific benchmarks.\nOn the challenging and large scale M5 benchmark, a real-world retail dataset, TSMixer\ndemonstrates superior performance compared to the state-of-the-art alternatives. Our results\nunderline the importance of efficiently utilizing cross-variate and auxiliary information for\nimproving the performance of time series forecasting. We present various analyses to shed\nlight into the capabilities of TSMixer. The design paradigms utilized in TSMixer are expected\nto open new horizons for deep learning-based time series forecasting. The implementation\nis available at:https://github.com/google-research/google-research/tree/master/\ntsmixer.\n\n[1 Introduction]\nTime series forecasting is a prevalent problem in numerous real-world use cases, such as for forecasting of\ndemand of products (Böse et al., 2017; Courty & Li, 1999), pandemic spread (Zhang & Nawata, 2018), and\ninflation rates (Capistrán et al., 2010). The forecastability of time series data often originates from three\nmajor aspects:\n\narXiv:2303.06053v5 [cs.LG] 11 Sep 2023\nPublished in Transactions on Machine Learning Research (09/2023)\nFigure 1: TSMixer for multivariate time series forecasting. The columns of the inputs means different\nfeatures/variates and the rows are time steps. The fully-connected operations are row-wise. TSMixer contains\ninterleaving time-mixing and feature-mixing MLPs to aggregate information. The number of mixer layer is\ndenoted asN. The time-mixing MLPs are shared across all features and the feature-mixing MLPs are shared\nacross all of the time steps. The design allow TSMixer to automatically adapt the use of both temporal and\ncross-variate information with limited number of parameters for superior generalization. The extension with\nauxiliary information is also explored in this paper.\n• Persistent temporal patterns: encompassing trends and seasonal patterns, e.g., long-term inflation,\nday-of-week effects;\n• Cross-variate information: correlations between different variables, e.g., an increase in blood pressure\nassociated with a rise in body weight;\n• Auxiliary features: comprising static features and future information, e.g., product categories and\npromotional events.\n\nPublished in Transactions on Machine Learning Research (09/2023)\nTraditional models, such as ARIMA (Box et al., 1970), are designed for univariate time series, where only\ntemporal information is available. Therefore, they face limitations when dealing with challenging real-world\ndata, which often contains complex cross-variate information and auxiliary features. In contrast, numerous\ndeep learning models, particularly Transformer-based models, have been proposed due to their capacity to\ncapture both complex temporal patterns and cross-variate dependencies (Gamboa, 2017; Li et al., 2019; Wen\net al., 2017; Zhou et al., 2021; Wu et al., 2021; Lim & Zohren, 2021; Liu et al., 2022a; Zhou et al., 2022b; Liu\net al., 2022b; Zhou et al., 2022a) .\nThe natural intuition is that multivariate models, such as those based on Transformer architectures, should be\nmore effective than univariate models due to their ability to leverage cross-variate information. However, Zeng\net al. (2023) revealed that this is not always the case – Transformer-based models can indeed be significantly\nworse than simple univariate temporal linear models on many commonly used forecasting benchmarks. The\nmultivariate models seem to suffer from overfitting especially when the target time series is not correlated\nwith other covariates. This surprising finding has raised two essential questions:\n1. Does cross-variate information truly provide a benefit for time series forecasting?\n2. When cross-variate information is not beneficial, can multivariate models still perform as well as\nunivariate models?\nTo address these questions, we begin by analyzing the effectiveness of temporal linear models. Our findings\nindicate that their time-step-dependent characteristics render temporal linear models great candidates for\nlearning temporal patterns under common assumptions. Consequently, we gradually increase the capacity of\nlinear models by\n1. stacking temporal linear models with non-linearities (TMix-Only),\n2. introducing cross-variate feed-forward layers (TSMixer).\nThe resulting TSMixer alternatively applies MLPs across time and feature dimensions, conceptually corresponding totime-mixing and feature-mixing operations, efficiently capturing both temporal patterns and\ncross-variate information, as illustrated in Fig. 1. The residual designs ensure that TSMixer retains the\ncapacity of temporal linear models while still being able to exploit cross-variate information.\nWe evaluate TSMixer on commonly used long-term forecasting datasets (Wu et al., 2021) where univariate\nmodels have outperformed multivariate models. Our ablation study demonstrates the effectiveness of stacking\ntemporal linear models and validates that cross-variate information is less beneficial on these popular datasets,\nexplaining the superior performance of univariate models. Even so, TSMixer is on par with state-of-the-art\nunivariate models and significantly outperforms other multivariate models.\nTo demonstrate the benefit of multivariate models, we further evaluate TSMixer on the challenging M5\nbenchmark, a large-scale retail dataset used in the M-competition (Makridakis et al., 2022). M5 contains crucial\ncross-variate interactions such as sell prices (Makridakis et al., 2022). The results show that cross-variate\ninformation indeed brings significant improvement, and TSMixer can effectively leverage this information.\nFurthermore, we propose a principle design to extend TSMixer to handle auxiliary information such as static\nfeatures and future time-varying features. It aligns the different types of features into the same shape then\napplied mixer layers on the concatenated features to leverage the interactions between them. In this more\npractical and challenging setting, TSMixer outperforms models that are popular in industrial applications,\nincluding DeepAR (Salinas et al. 2020, Amazon SageMaker) and TFT (Lim et al. 2021, Google Cloud Vertex),\ndemonstrating its strong potential for real world impact.\nWe summarize our contributions as below:\n• We analyze the effectiveness of state-of-the-art linear models and indicate that their time-stepdependent characteristics make them great candidates for learning temporal patterns under common\nassumptions.\n\nPublished in Transactions on Machine Learning Research (09/2023)\nTable 1: Recent works in time series forecasting. Category I is univariate time series forecasting; Category II\nis multivariate time series forecasting, and Category III is time series forecasting with auxiliary information.\nIn this work, we propose TSMixer for Category II. We also extend TSMixer to leverage auxiliary information\nincluding static and future time-varying features for Category III.\nCategory\nExtrapolating Consideration of Consideration of\nModelstemporal patternscross-variate informationauxiliary features\n(i.e. multivariateness)\nI ✔\nARIMA (Box et al., 1970)\nN-BEATS (Oreshkin et al., 2020)\nLTSF-Linear (Zeng et al., 2023)\nPatchTST (Nie et al., 2023)\nII ✔ ✔\nInformer (Zhou et al., 2021)\nAutoformer (Wu et al., 2021)\nPyraformer (Liu et al., 2022a)\nFEDformer (Zhou et al., 2022b)\nNS-Transformer (Liu et al., 2022b)\nFiLM (Zhou et al., 2022a)\nTSMixer(this work)\nIII ✔ ✔ ✔\nMQRNN (Wen et al., 2017)\nDSSM (Rangapuram et al., 2018)\nDeepAR (Salinas et al., 2020)\nTFT (Lim et al., 2021)\nTSMixer-Ext(this work)\n• We propose TSMixer, an innovative architecture which retains the capacity of linear models to\ncapture temporal patterns while still being able to exploit cross-variate information.\n• We point out the potential risk of evaluating multivariate models on common long-term forecasting\nbenchmarks.\n• Our empirical studies demonstrate that TSMixer is the first multivariate model which is on par with\nunivariate models on common benchmarks and achieves state-of-the-art on a large-scale industrial\napplication where cross-variate information is crucial.\n\n[4 Tsmixer Architecture]\nExpanding upon our finding that linear models can serve as strong candidates for capturing time dependencies,\nwe initially propose a natural enhancement by stacking linear models with non-linearities to form multi-layer\nperceptrons (MLPs). Common deep learning techniques, such as normalization and residual connections, are\napplied to facilitate efficient learning. However, this architecture does not take cross-variate information into\naccount.\nTo better leverage cross-variate information, we propose the application of MLPs in the time-domain and\nthe feature-domain in an alternating manner. The time-domain MLPs are shared across all of the features,\nwhile the feature-domain MLPs are shared across all of the time steps. This resulting model is akin to the\nMLP-Mixer architecture from computer vision (Tolstikhin et al., 2021), with time-domain and feature-domain\n\nPublished in Transactions on Machine Learning Research (09/2023)\nwt−2 wt−1 wt ft−2(x)ft−1(x) ft(x)\nxt−2 xt−1 xt\nxt+1\nxt+1 = ∑t\ni=1 wixi\nTime-step-dependent\nxt−2 xt−1 xt\nxt+1\nxt+1 = ∑t\ni=1 fi(x)xi\nData-dependent\nFigure 2: Illustrations of time-step-dependent and data-dependent models within a single forecasting time\nstep.\nFigure 3: The architecture of TMix-Only. It is similar to TSMixer but only applies time-mixing.\noperations representing time-mixing and feature-mixing operations, respectively. Consequently, we name our\nproposed architecture Time-Series Mixer (TSMixer).\nThe interleaving design between these two operations efficiently utilizes both temporal dependencies and\ncross-variate information while limiting computational complexity and model size. It allows TSMixer to use a\nlong lookback window (see Sec. 3), while maintaining the parameter growth in onlyO(L+ C) instead of\nO(LC) if fully-connected MLPs were used. To better understand the utility of cross-variate information and\nfeature-mixing, we also consider a simplified variant of TSMixer that only employs time-mixing, referred to\nas TMix-Only, which consists of a residual MLP shared across each variate, as illustrated in Fig. 3. We also\npresent the extension of TSMixer to scenarios where auxiliary information about the time series is available.\n\n[6 Conclusions]\nWe propose TSMixer, a novel architecture for time series forecasting that is designed using MLPs instead of\ncommonly used RNNs and attention mechanisms to obtain superior generalization with a simple architecture.\nOur results at a wide range of real-world time series forecasting tasks demonstrate that TSMixer is highly\neffective in both long-term forecasting benchmarks for multivariate time-series, and real-world large-scale\nretail demand forecasting tasks. Notably, TSMixer is the only multivariate model that is able to achieve\nsimilar performance to univariate models in long term time series forecasting benchmarks. The TSMixer\narchitecture has significant potential for further improvement and we believe it will be useful in a wide\nrange of time series forecasting tasks. Some of the potential future works include further exploring the\ninterpretability of TSMixer, as well as its scalability to even larger datasets. We hope this work will pave the\nway for more innovative architectures for time series forecasting.\n\nPublished in Transactions on Machine Learning Research (09/2023)",
"[ABSTRACT]\nLow-dimensional embeddings of nodes in large graphs have proved extremely\nuseful in a variety of prediction tasks, from content recommendation to identifying\nprotein functions. However, most existing approaches require that all nodes in the\ngraph are present during training of the embeddings; these previous approaches are\ninherently transductive and do not naturally generalize to unseen nodes. Here we\npresent GraphSAGE, a general inductive framework that leverages node feature\ninformation (e.g., text attributes) to efficiently generate node embeddings for\npreviously unseen data. Instead of training individual embeddings for each node,\nwe learn a function that generates embeddings by sampling and aggregating features\nfrom a node’s local neighborhood. Our algorithm outperforms strong baselines\non three inductive node-classification benchmarks: we classify the category of\nunseen nodes in evolving information graphs based on citation and Reddit post\ndata, and we show that our algorithm generalizes to completely unseen graphs\nusing a multi-graph dataset of protein-protein interactions.\n\n[1 Introduction]\nLow-dimensional vector embeddings of nodes in large graphs 1 have proved extremely useful as\nfeature inputs for a wide variety of prediction and graph analysis tasks [5, 11, 28, 35, 36]. The basic\nidea behind node embedding approaches is to use dimensionality reduction techniques to distill the\nhigh-dimensional information about a node’s graph neighborhood into a dense vector embedding.\nThese node embeddings can then be fed to downstream machine learning systems and aid in tasks\nsuch as node classification, clustering, and link prediction [11, 28, 35].\nHowever, previous works have focused on embedding nodes from a single fixed graph, and many\nreal-world applications require embeddings to be quickly generated for unseen nodes, or entirely new\n(sub)graphs. This inductive capability is essential for high-throughput, production machine learning\nsystems, which operate on evolving graphs and constantly encounter unseen nodes (e.g., posts on\nReddit, users and videos on Youtube). An inductive approach to generating node embeddings also\nfacilitates generalization across graphs with the same form of features: for example, one could train\nan embedding generator on protein-protein interaction graphs derived from a model organism, and\nthen easily produce node embeddings for data collected on new organisms using the trained model.\nThe inductive node embedding problem is especially difficult, compared to the transductive setting,\nbecause generalizing to unseen nodes requires “aligning” newly observed subgraphs to the node\nembeddings that the algorithm has already optimized on. An inductive framework must learn to\n∗The two first authors made equal contributions.\n1While it is common to refer to these data structures as social or biological networks, we use the term graph\nto avoid ambiguity with neural network terminology.\n31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.\narXiv:1706.02216v4 [cs.SI] 10 Sep 2018\nFigure 1: Visual illustration of the GraphSAGE sample and aggregate approach.\nrecognize structural properties of a node’s neighborhood that reveal both the node’s local role in the\ngraph, as well as its global position.\nMost existing approaches to generating node embeddings are inherently transductive. The majority\nof these approaches directly optimize the embeddings for each node using matrix-factorization-based\nobjectives, and do not naturally generalize to unseen data, since they make predictions on nodes in a\nsingle, fixed graph [5, 11, 23, 28, 35, 36, 37, 39]. These approaches can be modified to operate in an\ninductive setting (e.g., [28]), but these modifications tend to be computationally expensive, requiring\nadditional rounds of gradient descent before new predictions can be made. There are also recent\napproaches to learning over graph structures using convolution operators that offer promise as an\nembedding methodology [17]. So far, graph convolutional networks (GCNs) have only been applied\nin the transductive setting with fixed graphs [17, 18]. In this work we both extend GCNs to the task\nof inductive unsupervised learning and propose a framework that generalizes the GCN approach to\nuse trainable aggregation functions (beyond simple convolutions).\nPresent work. We propose a general framework, called GraphSAGE (SAmple and aggreGatE), for\ninductive node embedding. Unlike embedding approaches that are based on matrix factorization,\nwe leverage node features (e.g., text attributes, node profile information, node degrees) in order to\nlearn an embedding function that generalizes to unseen nodes. By incorporating node features in the\nlearning algorithm, we simultaneously learn the topological structure of each node’s neighborhood\nas well as the distribution of node features in the neighborhood. While we focus on feature-rich\ngraphs (e.g., citation data with text attributes, biological data with functional/molecular markers), our\napproach can also make use of structural features that are present in all graphs (e.g., node degrees).\nThus, our algorithm can also be applied to graphs without node features.\nInstead of training a distinct embedding vector for each node, we train a set of aggregator functions\nthat learn to aggregate feature information from a node’s local neighborhood (Figure 1). Each\naggregator function aggregates information from a different number of hops, or search depth, away\nfrom a given node. At test, or inference time, we use our trained system to generate embeddings for\nentirely unseen nodes by applying the learned aggregation functions. Following previous work on\ngenerating node embeddings, we design an unsupervised loss function that allows GraphSAGE to be\ntrained without task-specific supervision. We also show that GraphSAGE can be trained in a fully\nsupervised manner.\nWe evaluate our algorithm on three node-classification benchmarks, which test GraphSAGE’s ability\nto generate useful embeddings on unseen data. We use two evolving document graphs based on\ncitation data and Reddit post data (predicting paper and post categories, respectively), and a multigraph generalization experiment based on a dataset of protein-protein interactions (predicting protein\nfunctions). Using these benchmarks, we show that our approach is able to effectively generate\nrepresentations for unseen nodes and outperform relevant baselines by a significant margin: across\ndomains, our supervised approach improves classification F1-scores by an average of 51% compared\nto using node features alone and GraphSAGE consistently outperforms a strong, transductive baseline\n[28], despite this baseline taking ∼100×longer to run on unseen nodes. We also show that the new\naggregator architectures we propose provide significant gains (7.4% on average) compared to an\naggregator inspired by graph convolutional networks [17]. Lastly, we probe the expressive capability\nof our approach and show, through theoretical analysis, that GraphSAGE is capable of learning\nstructural information about a node’s role in a graph, despite the fact that it is inherently based on\nfeatures (Section 5).\n\n[3.3 Aggregator Architectures]\nUnlike machine learning over N-D lattices (e.g., sentences, images, or 3-D volumes), a node’s\nneighbors have no natural ordering; thus, the aggregator functions in Algorithm 1 must operate over\nan unordered set of vectors. Ideally, an aggregator function would be symmetric (i.e., invariant to\npermutations of its inputs) while still being trainable and maintaining high representational capacity.\nThe symmetry property of the aggregation function ensures that our neural network model can\nbe trained and applied to arbitrarily ordered node neighborhood feature sets. We examined three\ncandidate aggregator functions:\nMean aggregator. Our first candidate aggregator function is the mean operator, where we simply\ntake the elementwise mean of the vectors in {hk−1\nu ,∀u ∈N(v)}. The mean aggregator is nearly\nequivalent to the convolutional propagation rule used in the transductive GCN framework [17]. In\nparticular, we can derive an inductive variant of the GCN approach by replacing lines 4 and 5 in\nAlgorithm 1 with the following:4\nhk\nv ←σ(W ·MEAN ({hk−1\nv }∪{hk−1\nu ,∀u∈N(v)}). (2)\nWe call this modified mean-based aggregatorconvolutional since it is a rough, linear approximation of\na localized spectral convolution [17]. An important distinction between this convolutional aggregator\nand our other proposed aggregators is that it does not perform the concatenation operation in line\n5 of Algorithm 1—i.e., the convolutional aggregator does concatenate the node’s previous layer\nrepresentation hk−1\nv with the aggregated neighborhood vector hk\nN(v). This concatenation can be\nviewed as a simple form of a “skip connection” [13] between the different “search depths”, or “layers”\nof the GraphSAGE algorithm, and it leads to significant gains in performance (Section 4).\nLSTM aggregator. We also examined a more complex aggregator based on an LSTM architecture\n[14]. Compared to the mean aggregator, LSTMs have the advantage of larger expressive capability.\nHowever, it is important to note that LSTMs are not inherently symmetric (i.e., they are not permutation invariant), since they process their inputs in a sequential manner. We adapt LSTMs to operate on\nan unordered set by simply applying the LSTMs to a random permutation of the node’s neighbors.\n3Exploring non-uniform samplers is an important direction for future work.\n4Note that this differs from Kipf et al’s exact equation by a minor normalization constant [17].\n\nPooling aggregator. The final aggregator we examine is both symmetric and trainable. In this\npooling approach, each neighbor’s vector is independently fed through a fully-connected neural\nnetwork; following this transformation, an elementwise max-pooling operation is applied to aggregate\ninformation across the neighbor set:\nAGGREGATE pool\nk = max({σ\n(\nWpoolhk\nui + b\n)\n,∀ui ∈N(v)}), (3)\nwhere max denotes the element-wise max operator and σ is a nonlinear activation function. In\nprinciple, the function applied before the max pooling can be an arbitrarily deep multi-layer perceptron, but we focus on simple single-layer architectures in this work. This approach is inspired by\nrecent advancements in applying neural network architectures to learn over general point sets [29].\nIntuitively, the multi-layer perceptron can be thought of as a set of functions that compute features for\neach of the node representations in the neighbor set. By applying the max-pooling operator to each of\nthe computed features, the model effectively captures different aspects of the neighborhood set. Note\nalso that, in principle, any symmetric vector function could be used in place of the max operator\n(e.g., an element-wise mean). We found no significant difference between max- and mean-pooling in\ndevelopments test and thus focused on max-pooling for the rest of our experiments.\n\n[6 Conclusion]\nWe introduced a novel approach that allows embeddings to be efficiently generated for unseen nodes.\nGraphSAGE consistently outperforms state-of-the-art baselines, effectively trades off performance\nand runtime by sampling node neighborhoods, and our theoretical analysis provides insight into\nhow our approach can learn about local graph structures. A number of extensions and potential\nimprovements are possible, such as extending GraphSAGE to incorporate directed or multi-modal\ngraphs. A particularly interesting direction for future work is exploring non-uniform neighborhood\nsampling functions, and perhaps even learning these functions as part of the GraphSAGE optimization.\nAcknowledgments\nThe authors thank Austin Benson, Aditya Grover, Bryan He, Dan Jurafsky, Alex Ratner, Marinka\nZitnik, and Daniel Selsam for their helpful discussions and comments on early drafts. The authors\nwould also like to thank Ben Johnson for his many useful questions and comments on our code and\nNikhil Mehta and Yuhui Ding for catching some minor errors in a previous version of the appendix.\nThis research has been supported in part by NSF IIS-1149837, DARPA SIMPLEX, Stanford Data\nScience Initiative, Huawei, and Chan Zuckerberg Biohub. WLH was also supported by the SAP\nStanford Graduate Fellowship and an NSERC PGS-D grant. The views and conclusions expressed\nin this material are those of the authors and should not be interpreted as necessarily representing\nthe official policies or endorsements, either expressed or implied, of the above funding agencies,\ncorporations, or the U.S. and Canadian governments."
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4] 
