graziul commited on
Commit
df28a66
·
verified ·
1 Parent(s): 5f58cd5

feat: collapsible expansion, canonical formalism pages, citation enrichment

Browse files
Files changed (1) hide show
  1. formalisms.yaml +815 -0
formalisms.yaml ADDED
@@ -0,0 +1,815 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ formalisms:
2
+ - id: pca
3
+ name: Principal Component Analysis
4
+ year: 1901
5
+ origin: Pearson; Hotelling (1933)
6
+ signature:
7
+ operation: project
8
+ domain: vector
9
+ codomain: vector
10
+ objective_family: none
11
+ meso_type: linear_projection
12
+ macro_type: eigenvalue_problem
13
+ canonical_reference: Pearson, K. (1901). On lines and planes of closest fit.
14
+ status: seed
15
+ - id: cca
16
+ name: Canonical Correlation Analysis
17
+ year: 1936
18
+ origin: Hotelling
19
+ signature:
20
+ operation: project
21
+ domain: vector
22
+ codomain: vector
23
+ objective_family: correlation
24
+ meso_type: joint_embedding
25
+ macro_type: eigenvalue_problem
26
+ canonical_reference: Hotelling, H. (1936). Relations between two sets of variates.
27
+ status: seed
28
+ - id: kernel_cca
29
+ name: Kernel Canonical Correlation Analysis
30
+ year: 2002
31
+ origin: Bach & Jordan
32
+ signature:
33
+ operation: project
34
+ domain: vector
35
+ codomain: vector
36
+ objective_family: correlation
37
+ meso_type: kernel_method
38
+ macro_type: eigenvalue_problem
39
+ canonical_reference: Bach, F.R. & Jordan, M.I. (2002). Kernel independent component analysis. JMLR.
40
+ status: seed
41
+ - id: ica
42
+ name: Independent Component Analysis
43
+ year: 1994
44
+ origin: Comon
45
+ signature:
46
+ operation: decompose
47
+ domain: vector
48
+ codomain: vector
49
+ objective_family: information
50
+ meso_type: linear_projection
51
+ macro_type: optimization
52
+ canonical_reference: Comon, P. (1994). Independent component analysis, a new concept?
53
+ status: seed
54
+ - id: fisher_lda
55
+ name: Fisher Linear Discriminant Analysis
56
+ year: 1936
57
+ origin: Fisher
58
+ signature:
59
+ operation: project
60
+ domain: vector
61
+ codomain: vector
62
+ objective_family: none
63
+ meso_type: linear_projection
64
+ macro_type: eigenvalue_problem
65
+ canonical_reference: Fisher, R.A. (1936). The use of multiple measurements in taxonomic problems.
66
+ status: seed
67
+ - id: mds
68
+ name: Multi-Dimensional Scaling
69
+ year: 1952
70
+ origin: Torgerson
71
+ signature:
72
+ operation: project
73
+ domain: matrix
74
+ codomain: vector
75
+ objective_family: none
76
+ meso_type: spectral_method
77
+ macro_type: eigenvalue_problem
78
+ canonical_reference: 'Torgerson, W.S. (1952). Multidimensional scaling: I. Theory and method.'
79
+ status: seed
80
+ - id: isomap
81
+ name: Isometric Feature Mapping
82
+ year: 2000
83
+ origin: Tenenbaum, de Silva & Langford
84
+ signature:
85
+ operation: project
86
+ domain: manifold
87
+ codomain: vector
88
+ objective_family: none
89
+ meso_type: spectral_method
90
+ macro_type: eigenvalue_problem
91
+ canonical_reference: Tenenbaum, J.B., de Silva, V., & Langford, J.C. (2000). A global geometric framework for nonlinear
92
+ dimensionality reduction. Science.
93
+ status: seed
94
+ - id: lle
95
+ name: Locally Linear Embedding
96
+ year: 2000
97
+ origin: Roweis & Saul
98
+ signature:
99
+ operation: project
100
+ domain: manifold
101
+ codomain: vector
102
+ objective_family: none
103
+ meso_type: spectral_method
104
+ macro_type: eigenvalue_problem
105
+ canonical_reference: Roweis, S.T. & Saul, L.K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science.
106
+ status: seed
107
+ - id: laplacian_eigenmaps
108
+ name: Laplacian Eigenmaps
109
+ year: 2003
110
+ origin: Belkin & Niyogi
111
+ signature:
112
+ operation: project
113
+ domain: manifold
114
+ codomain: vector
115
+ objective_family: energy
116
+ meso_type: spectral_method
117
+ macro_type: eigenvalue_problem
118
+ canonical_reference: Belkin, M. & Niyogi, P. (2003). Laplacian eigenmaps for dimensionality reduction and data representation.
119
+ Neural Computation.
120
+ researchor_artifact_id: laplacian_matrix
121
+ status: seed
122
+ - id: svd
123
+ name: Singular Value Decomposition
124
+ year: 1873
125
+ origin: Beltrami; Jordan; Golub & Reinsch
126
+ signature:
127
+ operation: decompose
128
+ domain: matrix
129
+ codomain: matrix
130
+ objective_family: none
131
+ meso_type: linear_projection
132
+ macro_type: eigenvalue_problem
133
+ canonical_reference: Golub, G.H. & Reinsch, C. (1970). Singular value decomposition and least squares solutions.
134
+ researchor_artifact_id: singular_value_decomposition
135
+ status: seed
136
+ - id: nmf
137
+ name: Non-negative Matrix Factorization
138
+ year: 1999
139
+ origin: Lee & Seung
140
+ signature:
141
+ operation: decompose
142
+ domain: matrix
143
+ codomain: matrix
144
+ objective_family: divergence
145
+ meso_type: linear_projection
146
+ macro_type: optimization
147
+ canonical_reference: Lee, D.D. & Seung, H.S. (1999). Learning the parts of objects by non-negative matrix factorization.
148
+ Nature.
149
+ status: seed
150
+ canonical_arxiv_id: 0408058
151
+ - id: kernel_pca
152
+ name: Kernel Principal Component Analysis
153
+ year: 1998
154
+ origin: Schölkopf, Smola & Müller
155
+ signature:
156
+ operation: project
157
+ domain: vector
158
+ codomain: vector
159
+ objective_family: none
160
+ meso_type: kernel_method
161
+ macro_type: eigenvalue_problem
162
+ canonical_reference: Schölkopf, B., Smola, A., & Müller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue
163
+ problem. Neural Computation.
164
+ status: seed
165
+ - id: kernel_ridge_regression
166
+ name: Kernel Ridge Regression
167
+ year: 1998
168
+ origin: Saunders, Gammerman & Vovk; extended by many
169
+ signature:
170
+ operation: transform
171
+ domain: vector
172
+ codomain: scalar
173
+ objective_family: none
174
+ meso_type: kernel_method
175
+ macro_type: optimization
176
+ canonical_reference: Saunders, C., Gammerman, A., & Vovk, V. (1998). Ridge regression learning algorithm in dual variables.
177
+ ICML.
178
+ status: seed
179
+ - id: gaussian_process
180
+ name: Gaussian Process Regression
181
+ year: 1996
182
+ origin: Williams & Rasmussen; roots in kriging (Matheron 1963)
183
+ signature:
184
+ operation: transform
185
+ domain: vector
186
+ codomain: distribution
187
+ objective_family: likelihood
188
+ meso_type: kernel_method
189
+ macro_type: statistical_inference
190
+ canonical_reference: Rasmussen, C.E. & Williams, C.K.I. (2006). Gaussian Processes for Machine Learning. MIT Press.
191
+ status: seed
192
+ - id: svm
193
+ name: Support Vector Machine
194
+ year: 1995
195
+ origin: Cortes & Vapnik
196
+ signature:
197
+ operation: transform
198
+ domain: vector
199
+ codomain: assignment
200
+ objective_family: none
201
+ meso_type: kernel_method
202
+ macro_type: optimization
203
+ canonical_reference: Cortes, C. & Vapnik, V. (1995). Support-vector networks. Machine Learning.
204
+ status: seed
205
+ - id: mmd
206
+ name: Maximum Mean Discrepancy
207
+ year: 2006
208
+ origin: Gretton, Borgwardt, Rasch, Schölkopf & Smola
209
+ signature:
210
+ operation: match
211
+ domain: distribution
212
+ codomain: scalar
213
+ objective_family: none
214
+ meso_type: kernel_method
215
+ macro_type: statistical_inference
216
+ canonical_reference: Gretton, A. et al. (2012). A kernel two-sample test. JMLR.
217
+ status: seed
218
+ - id: kl_divergence_min
219
+ name: KL Divergence Minimization
220
+ year: 1951
221
+ origin: Kullback & Leibler
222
+ signature:
223
+ operation: minimize
224
+ domain: distribution
225
+ codomain: distribution
226
+ objective_family: divergence
227
+ meso_type: information_geometry
228
+ macro_type: optimization
229
+ canonical_reference: Kullback, S. & Leibler, R.A. (1951). On information and sufficiency.
230
+ researchor_artifact_id: relative_entropy
231
+ status: seed
232
+ - id: mutual_info_max
233
+ name: Mutual Information Maximization
234
+ year: 1948
235
+ origin: Shannon; McGill (1954)
236
+ signature:
237
+ operation: maximize
238
+ domain: distribution
239
+ codomain: scalar
240
+ objective_family: information
241
+ meso_type: information_geometry
242
+ macro_type: optimization
243
+ canonical_reference: Shannon, C.E. (1948). A mathematical theory of communication.
244
+ researchor_artifact_id: mutual_information
245
+ status: seed
246
+ - id: cross_entropy_min
247
+ name: Cross-Entropy Minimization
248
+ year: 1948
249
+ origin: Shannon; Good (1956)
250
+ signature:
251
+ operation: minimize
252
+ domain: distribution
253
+ codomain: distribution
254
+ objective_family: divergence
255
+ meso_type: information_geometry
256
+ macro_type: optimization
257
+ canonical_reference: Good, I.J. (1956). The population frequencies of species and the estimation of population parameters.
258
+ researchor_artifact_id: entropy
259
+ status: seed
260
+ - id: infonce
261
+ name: InfoNCE / Contrastive Estimation
262
+ year: 2018
263
+ origin: van den Oord, Li & Vinyals
264
+ signature:
265
+ operation: maximize
266
+ domain: vector
267
+ codomain: scalar
268
+ objective_family: information
269
+ meso_type: joint_embedding
270
+ macro_type: optimization
271
+ canonical_reference: van den Oord, A., Li, Y., & Vinyals, O. (2018). Representation learning with contrastive predictive
272
+ coding. arXiv:1807.03748.
273
+ status: seed
274
+ canonical_arxiv_id: '1807.03748'
275
+ - id: js_divergence
276
+ name: Jensen-Shannon Divergence
277
+ year: 1991
278
+ origin: Lin; extended from KL
279
+ signature:
280
+ operation: match
281
+ domain: distribution
282
+ codomain: scalar
283
+ objective_family: divergence
284
+ meso_type: information_geometry
285
+ macro_type: statistical_inference
286
+ canonical_reference: Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Trans. Info. Theory.
287
+ status: seed
288
+ - id: elbo
289
+ name: Evidence Lower BOund Maximization
290
+ year: 1999
291
+ origin: Jordan, Ghahramani, Jaakkola & Saul; variational Bayes
292
+ signature:
293
+ operation: maximize
294
+ domain: distribution
295
+ codomain: distribution
296
+ objective_family: divergence
297
+ meso_type: variational
298
+ macro_type: optimization
299
+ canonical_reference: Jordan, M.I. et al. (1999). An introduction to variational methods for graphical models. Machine Learning.
300
+ status: seed
301
+ canonical_arxiv_id: '1312.6114'
302
+ - id: variational_inference
303
+ name: Variational Inference
304
+ year: 1990
305
+ origin: Hinton & van Camp; Jordan et al.; Wainwright & Jordan
306
+ signature:
307
+ operation: minimize
308
+ domain: distribution
309
+ codomain: distribution
310
+ objective_family: divergence
311
+ meso_type: variational
312
+ macro_type: statistical_inference
313
+ canonical_reference: Wainwright, M.J. & Jordan, M.I. (2008). Graphical models, exponential families, and variational inference.
314
+ Foundations & Trends in ML.
315
+ status: seed
316
+ - id: mean_field
317
+ name: Mean-Field Approximation
318
+ year: 1937
319
+ origin: Landau; Weiss; applied to stat mech and later VI
320
+ signature:
321
+ operation: minimize
322
+ domain: distribution
323
+ codomain: distribution
324
+ objective_family: divergence
325
+ meso_type: mean_field
326
+ macro_type: statistical_inference
327
+ canonical_reference: Parisi, G. (1988). Statistical Field Theory. Addison-Wesley.
328
+ status: seed
329
+ - id: mle
330
+ name: Maximum Likelihood Estimation
331
+ year: 1922
332
+ origin: Fisher
333
+ signature:
334
+ operation: maximize
335
+ domain: distribution
336
+ codomain: scalar
337
+ objective_family: likelihood
338
+ meso_type: probabilistic_inference
339
+ macro_type: statistical_inference
340
+ canonical_reference: Fisher, R.A. (1922). On the mathematical foundations of theoretical statistics.
341
+ status: seed
342
+ - id: map_estimation
343
+ name: Maximum A Posteriori Estimation
344
+ year: 1763
345
+ origin: Bayes; Laplace
346
+ signature:
347
+ operation: maximize
348
+ domain: distribution
349
+ codomain: scalar
350
+ objective_family: likelihood
351
+ meso_type: probabilistic_inference
352
+ macro_type: statistical_inference
353
+ canonical_reference: Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer.
354
+ status: seed
355
+ - id: em_algorithm
356
+ name: Expectation-Maximization
357
+ year: 1977
358
+ origin: Dempster, Laird & Rubin
359
+ signature:
360
+ operation: maximize
361
+ domain: distribution
362
+ codomain: distribution
363
+ objective_family: likelihood
364
+ meso_type: variational
365
+ macro_type: optimization
366
+ canonical_reference: Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the
367
+ EM algorithm. JRSS-B.
368
+ status: seed
369
+ - id: mcmc
370
+ name: Markov Chain Monte Carlo
371
+ year: 1953
372
+ origin: Metropolis, Rosenbluth, Rosenbluth, Teller & Teller; Hastings (1970)
373
+ signature:
374
+ operation: sample
375
+ domain: distribution
376
+ codomain: sequence
377
+ objective_family: none
378
+ meso_type: probabilistic_inference
379
+ macro_type: stochastic_process
380
+ canonical_reference: Metropolis, N. et al. (1953). Equation of state calculations by fast computing machines.
381
+ researchor_artifact_id: markov_chain
382
+ status: seed
383
+ - id: empirical_bayes
384
+ name: Empirical Bayes / Type-II Maximum Likelihood
385
+ year: 1955
386
+ origin: Robbins; Efron & Morris
387
+ signature:
388
+ operation: maximize
389
+ domain: distribution
390
+ codomain: distribution
391
+ objective_family: likelihood
392
+ meso_type: probabilistic_inference
393
+ macro_type: statistical_inference
394
+ canonical_reference: Robbins, H. (1955). An empirical Bayes approach to statistics. Proc. Third Berkeley Symp.
395
+ status: seed
396
+ - id: normalizing_flow
397
+ name: Normalizing Flow
398
+ year: 2015
399
+ origin: Rezende & Mohamed; Dinh, Krueger & Bengio (NICE 2014)
400
+ signature:
401
+ operation: transform
402
+ domain: distribution
403
+ codomain: distribution
404
+ objective_family: likelihood
405
+ meso_type: variational
406
+ macro_type: statistical_inference
407
+ canonical_reference: Rezende, D.J. & Mohamed, S. (2015). Variational inference with normalizing flows. ICML.
408
+ status: seed
409
+ canonical_arxiv_id: '1505.05770'
410
+ - id: boltzmann_distribution
411
+ name: Boltzmann / Gibbs Distribution
412
+ year: 1868
413
+ origin: Boltzmann; Gibbs (1902)
414
+ signature:
415
+ operation: sample
416
+ domain: scalar_field
417
+ codomain: distribution
418
+ objective_family: energy
419
+ meso_type: energy_model
420
+ macro_type: hamiltonian_system
421
+ canonical_reference: Gibbs, J.W. (1902). Elementary Principles in Statistical Mechanics.
422
+ researchor_mental_model_id: ergodicity
423
+ status: seed
424
+ - id: free_energy_min
425
+ name: Free Energy Minimization
426
+ year: 1873
427
+ origin: Helmholtz; Gibbs
428
+ signature:
429
+ operation: minimize
430
+ domain: distribution
431
+ codomain: distribution
432
+ objective_family: energy
433
+ meso_type: energy_model
434
+ macro_type: hamiltonian_system
435
+ canonical_reference: Helmholtz, H. (1882). Die Thermodynamik chemischer Vorgänge.
436
+ researchor_mental_model_id: phase_transitions
437
+ status: seed
438
+ - id: hopfield_network
439
+ name: Hopfield Network / Spin Glass
440
+ year: 1982
441
+ origin: Hopfield; Sherrington & Kirkpatrick (1975 spin glass)
442
+ signature:
443
+ operation: minimize
444
+ domain: vector
445
+ codomain: assignment
446
+ objective_family: energy
447
+ meso_type: spin_system
448
+ macro_type: hamiltonian_system
449
+ canonical_reference: Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective computational
450
+ abilities. PNAS.
451
+ researchor_mental_model_id: attractors_and_basins
452
+ status: seed
453
+ - id: renormalization_group
454
+ name: Renormalization Group
455
+ year: 1971
456
+ origin: Wilson; Kadanoff (block-spin 1966)
457
+ signature:
458
+ operation: transform
459
+ domain: scalar_field
460
+ codomain: scalar_field
461
+ objective_family: none
462
+ meso_type: energy_model
463
+ macro_type: hamiltonian_system
464
+ canonical_reference: Wilson, K.G. (1971). Renormalization group and critical phenomena. Phys. Rev. B.
465
+ researchor_mental_model_id: scale_invariance
466
+ status: seed
467
+ - id: langevin_dynamics
468
+ name: Langevin Dynamics
469
+ year: 1908
470
+ origin: Langevin; adapted to sampling by Parisi (1981)
471
+ signature:
472
+ operation: sample
473
+ domain: scalar_field
474
+ codomain: distribution
475
+ objective_family: energy
476
+ meso_type: dynamical_system
477
+ macro_type: stochastic_process
478
+ canonical_reference: Langevin, P. (1908). Sur la théorie du mouvement brownien.
479
+ researchor_artifact_id: brownian_motion
480
+ status: seed
481
+ - id: hamiltonian_monte_carlo
482
+ name: Hamiltonian Monte Carlo
483
+ year: 1987
484
+ origin: Duane, Kennedy, Pendleton & Roweth; Neal (2011 MCMC handbook)
485
+ signature:
486
+ operation: sample
487
+ domain: distribution
488
+ codomain: sequence
489
+ objective_family: energy
490
+ meso_type: dynamical_system
491
+ macro_type: hamiltonian_system
492
+ canonical_reference: Neal, R.M. (2011). MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo.
493
+ status: seed
494
+ - id: diffusion_sde
495
+ name: Diffusion Process / Reverse-Time SDE
496
+ year: 2015
497
+ origin: Sohl-Dickstein et al.; Ho, Jain & Abbeel (DDPM 2020); Song et al. (SDE 2021)
498
+ signature:
499
+ operation: sample
500
+ domain: distribution
501
+ codomain: distribution
502
+ objective_family: none
503
+ meso_type: diffusion_process
504
+ macro_type: stochastic_process
505
+ canonical_reference: Song, Y. et al. (2021). Score-based generative modeling through stochastic differential equations.
506
+ ICLR.
507
+ researchor_artifact_id: brownian_motion
508
+ researchor_mental_model_id: irreversibility
509
+ status: seed
510
+ canonical_arxiv_id: '2011.13456'
511
+ - id: wasserstein_distance
512
+ name: Wasserstein Distance / Earth Mover's Distance
513
+ year: 1781
514
+ origin: Monge; Kantorovich (1942)
515
+ signature:
516
+ operation: match
517
+ domain: distribution
518
+ codomain: scalar
519
+ objective_family: none
520
+ meso_type: optimal_transport
521
+ macro_type: optimization
522
+ canonical_reference: Kantorovich, L.V. (1942). On the translocation of masses.
523
+ status: seed
524
+ - id: sinkhorn_algorithm
525
+ name: Sinkhorn-Knopp Algorithm / Entropic OT
526
+ year: 1967
527
+ origin: Sinkhorn & Knopp; Cuturi (2013) for ML
528
+ signature:
529
+ operation: match
530
+ domain: distribution
531
+ codomain: matrix
532
+ objective_family: divergence
533
+ meso_type: optimal_transport
534
+ macro_type: optimization
535
+ canonical_reference: 'Cuturi, M. (2013). Sinkhorn distances: lightspeed computation of optimal transport. NeurIPS.'
536
+ status: seed
537
+ canonical_arxiv_id: '1306.0895'
538
+ - id: kantorovich_dual
539
+ name: Kantorovich Duality
540
+ year: 1942
541
+ origin: Kantorovich
542
+ signature:
543
+ operation: maximize
544
+ domain: distribution
545
+ codomain: scalar
546
+ objective_family: none
547
+ meso_type: optimal_transport
548
+ macro_type: optimization
549
+ canonical_reference: 'Villani, C. (2008). Optimal Transport: Old and New. Springer.'
550
+ status: seed
551
+ - id: gradient_descent
552
+ name: Gradient Descent
553
+ year: 1847
554
+ origin: Cauchy
555
+ signature:
556
+ operation: minimize
557
+ domain: scalar_field
558
+ codomain: vector
559
+ objective_family: none
560
+ meso_type: none
561
+ macro_type: optimization
562
+ canonical_reference: Cauchy, A. (1847). Méthode générale pour la résolution des systèmes d'équations simultanées.
563
+ researchor_artifact_id: gradient_descent
564
+ status: seed
565
+ - id: sgd
566
+ name: Stochastic Gradient Descent
567
+ year: 1951
568
+ origin: Robbins & Monro
569
+ signature:
570
+ operation: minimize
571
+ domain: scalar_field
572
+ codomain: vector
573
+ objective_family: none
574
+ meso_type: none
575
+ macro_type: optimization
576
+ canonical_reference: Robbins, H. & Monro, S. (1951). A stochastic approximation method.
577
+ researchor_artifact_id: stochastic_gradient_descent
578
+ status: seed
579
+ - id: lagrange_multiplier
580
+ name: Constrained Optimization (Lagrange Multiplier)
581
+ year: 1788
582
+ origin: Lagrange
583
+ signature:
584
+ operation: minimize
585
+ domain: scalar_field
586
+ codomain: vector
587
+ objective_family: none
588
+ meso_type: none
589
+ macro_type: optimization
590
+ canonical_reference: Lagrange, J.-L. (1788). Mécanique Analytique.
591
+ status: seed
592
+ - id: proximal_gradient
593
+ name: Proximal Gradient Method
594
+ year: 2005
595
+ origin: Combettes & Wajs; Beck & Teboulle (FISTA 2009)
596
+ signature:
597
+ operation: minimize
598
+ domain: scalar_field
599
+ codomain: vector
600
+ objective_family: none
601
+ meso_type: none
602
+ macro_type: optimization
603
+ canonical_reference: Beck, A. & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm. SIAM J. Imaging
604
+ Sci.
605
+ status: seed
606
+ - id: adam
607
+ name: Adam Optimizer
608
+ year: 2014
609
+ origin: Kingma & Ba
610
+ signature:
611
+ operation: minimize
612
+ domain: scalar_field
613
+ codomain: vector
614
+ objective_family: none
615
+ meso_type: none
616
+ macro_type: optimization
617
+ canonical_reference: 'Kingma, D.P. & Ba, J. (2015). Adam: a method for stochastic optimization. ICLR.'
618
+ status: seed
619
+ canonical_arxiv_id: '1412.6980'
620
+ - id: softmax_attention
621
+ name: Softmax Attention / Weighted Aggregation
622
+ year: 2014
623
+ origin: Bahdanau, Cho & Bengio; Vaswani et al. (Transformer 2017)
624
+ signature:
625
+ operation: aggregate
626
+ domain: sequence
627
+ codomain: vector
628
+ objective_family: none
629
+ meso_type: none
630
+ macro_type: none
631
+ canonical_reference: Vaswani, A. et al. (2017). Attention is all you need. NeurIPS.
632
+ status: seed
633
+ canonical_arxiv_id: '1706.03762'
634
+ - id: residual_connection
635
+ name: Residual Connection / Skip Connection
636
+ year: 2016
637
+ origin: He, Zhang, Ren & Sun; earlier in Hochreiter & Schmidhuber (LSTM 1997)
638
+ signature:
639
+ operation: transform
640
+ domain: vector
641
+ codomain: vector
642
+ objective_family: none
643
+ meso_type: none
644
+ macro_type: none
645
+ canonical_reference: He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. CVPR.
646
+ status: seed
647
+ canonical_arxiv_id: '1512.03385'
648
+ - id: batch_normalization
649
+ name: Batch Normalization
650
+ year: 2015
651
+ origin: Ioffe & Szegedy
652
+ signature:
653
+ operation: transform
654
+ domain: vector
655
+ codomain: vector
656
+ objective_family: none
657
+ meso_type: none
658
+ macro_type: none
659
+ canonical_reference: 'Ioffe, S. & Szegedy, C. (2015). Batch normalization: accelerating deep network training. ICML.'
660
+ status: seed
661
+ canonical_arxiv_id: '1502.03167'
662
+ - id: vae
663
+ name: Variational Autoencoder
664
+ year: 2013
665
+ origin: Kingma & Welling; Rezende, Mohamed & Wierstra
666
+ signature:
667
+ operation: transform
668
+ domain: vector
669
+ codomain: distribution
670
+ objective_family: divergence
671
+ meso_type: variational
672
+ macro_type: statistical_inference
673
+ canonical_reference: Kingma, D.P. & Welling, M. (2014). Auto-encoding variational Bayes. ICLR.
674
+ status: seed
675
+ canonical_arxiv_id: '1312.6114'
676
+ - id: gan
677
+ name: Generative Adversarial Network
678
+ year: 2014
679
+ origin: Goodfellow et al.
680
+ signature:
681
+ operation: minimize
682
+ domain: distribution
683
+ codomain: distribution
684
+ objective_family: adversarial
685
+ meso_type: game_theoretic
686
+ macro_type: optimization
687
+ canonical_reference: Goodfellow, I.J. et al. (2014). Generative adversarial nets. NeurIPS.
688
+ researchor_mental_model_id: nash_equilibrium
689
+ status: seed
690
+ canonical_arxiv_id: '1406.2661'
691
+ - id: contrastive_learning
692
+ name: Contrastive Learning
693
+ year: 2005
694
+ origin: Chopra, Hadsell & LeCun (Siamese, 2005); Hadsell et al. (dimensionality reduction 2006)
695
+ signature:
696
+ operation: minimize
697
+ domain: vector
698
+ codomain: scalar
699
+ objective_family: energy
700
+ meso_type: joint_embedding
701
+ macro_type: optimization
702
+ canonical_reference: Hadsell, R., Chopra, S., & LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping.
703
+ CVPR.
704
+ researchor_mental_model_id: signal_vs_noise
705
+ status: seed
706
+ - id: knowledge_distillation
707
+ name: Knowledge Distillation
708
+ year: 2015
709
+ origin: Hinton, Vinyals & Dean
710
+ signature:
711
+ operation: minimize
712
+ domain: distribution
713
+ codomain: distribution
714
+ objective_family: divergence
715
+ meso_type: none
716
+ macro_type: optimization
717
+ canonical_reference: Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. NeurIPS workshop.
718
+ status: seed
719
+ canonical_arxiv_id: '1503.02531'
720
+ - id: layer_normalization
721
+ name: Layer Normalization
722
+ year: 2016
723
+ origin: Ba, Kiros & Hinton
724
+ signature:
725
+ operation: transform
726
+ domain: vector
727
+ codomain: vector
728
+ objective_family: none
729
+ meso_type: none
730
+ macro_type: none
731
+ canonical_reference: Ba, J.L., Kiros, J.R., & Hinton, G.E. (2016). Layer normalization. arXiv:1607.06450.
732
+ status: seed
733
+ canonical_arxiv_id: '1607.06450'
734
+ - id: ridge_regression
735
+ name: Ridge Regression (L2 Regularization)
736
+ year: 1970
737
+ origin: Hoerl & Kennard; Tikhonov (1943)
738
+ signature:
739
+ operation: minimize
740
+ domain: vector
741
+ codomain: scalar
742
+ objective_family: none
743
+ meso_type: none
744
+ macro_type: optimization
745
+ canonical_reference: 'Hoerl, A.E. & Kennard, R.W. (1970). Ridge regression: biased estimation for nonorthogonal problems.'
746
+ status: seed
747
+ - id: lasso
748
+ name: LASSO (L1 Regularization)
749
+ year: 1996
750
+ origin: Tibshirani
751
+ signature:
752
+ operation: minimize
753
+ domain: vector
754
+ codomain: scalar
755
+ objective_family: none
756
+ meso_type: none
757
+ macro_type: optimization
758
+ canonical_reference: Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. JRSS-B.
759
+ status: seed
760
+ - id: elastic_net
761
+ name: Elastic Net
762
+ year: 2005
763
+ origin: Zou & Hastie
764
+ signature:
765
+ operation: minimize
766
+ domain: vector
767
+ codomain: scalar
768
+ objective_family: none
769
+ meso_type: none
770
+ macro_type: optimization
771
+ canonical_reference: Zou, H. & Hastie, T. (2005). Regularization and variable selection via the elastic net. JRSS-B.
772
+ status: seed
773
+ - id: spectral_clustering
774
+ name: Spectral Clustering
775
+ year: 2000
776
+ origin: Shi & Malik (normalized cuts); Ng, Jordan & Weiss (2002)
777
+ signature:
778
+ operation: decompose
779
+ domain: graph
780
+ codomain: assignment
781
+ objective_family: none
782
+ meso_type: spectral_method
783
+ macro_type: eigenvalue_problem
784
+ canonical_reference: 'Ng, A.Y., Jordan, M.I., & Weiss, Y. (2002). On spectral clustering: analysis and an algorithm. NeurIPS.'
785
+ researchor_artifact_id: laplacian_matrix
786
+ status: seed
787
+ - id: pagerank
788
+ name: PageRank
789
+ year: 1998
790
+ origin: Page, Brin, Motwani & Winograd
791
+ signature:
792
+ operation: propagate
793
+ domain: graph
794
+ codomain: vector
795
+ objective_family: none
796
+ meso_type: spectral_method
797
+ macro_type: eigenvalue_problem
798
+ canonical_reference: 'Page, L. et al. (1999). The PageRank citation ranking: bringing order to the web.'
799
+ researchor_artifact_id: adjacency_matrix
800
+ researchor_mental_model_id: network_centrality
801
+ status: seed
802
+ - id: fourier_transform
803
+ name: Fourier Transform / Spectral Decomposition
804
+ year: 1822
805
+ origin: Fourier; Cooley & Tukey (FFT 1965)
806
+ signature:
807
+ operation: decompose
808
+ domain: sequence
809
+ codomain: sequence
810
+ objective_family: none
811
+ meso_type: spectral_method
812
+ macro_type: none
813
+ canonical_reference: Fourier, J.B.J. (1822). Théorie analytique de la chaleur.
814
+ researchor_mental_model_id: signal_vs_noise
815
+ status: seed