Adapting this model so that it produces a statistical distribution rather than a vector-valued embedding?
#1
by mathlawguy - opened
I am wondering if anyone has considered how to adapt this model so that it represents the embedding of a sentence not as a vector but as a statistical distribution, perhaps a multivariate normal distribution with a diagonal covariance matrix. See https://arxiv.org/abs/1412.6623. Something more sophisticated would be even better. Either way, a distributional embedding would provide some quantification as to the possible meanings a passage of text might have.
Also, I appreciate that this is likely not the best place to post this more conceptual question. But I like this small model. If anyone has suggestions on a better forum, please advise.