Two ways of aligning molecular depictions into a frozen chemistry model’s embedding space, contrastive and predictive, compared on one data and evaluation stack with a pre-registered prediction.
I aligned a vision model into a frozen chemistry model’s embedding space two ways, contrastive and predictive, and held everything else identical. Four things surprised me, including a metric defect that produced exactly the result I had registered in advance.
SpeechT5 proposes a unified encoder-decoder pre-training framework that jointly learns from unlabeled speech and text data, achieving strong results on ASR, TTS, speech translation, voice conversion, speech enhancement, and speaker identification.
InstructMol integrates a pre-trained molecular graph encoder (MoleculeSTM) with a Vicuna-7B LLM using a linear projector. It employs a two-stage training process (alignment pre-training followed by task-specific instruction tuning with LoRA) to excel at property prediction, description generation, and reaction analysis.
A ground-up PyTorch Word2Vec. Tensorizing the Huffman tree turns pointer-chasing Hierarchical Softmax into dense GPU ops compatible with torch.compile.
We provide the first known analytical solution to Word2Vec’s softmax skip-gram objective, introducing the Independent Frequencies Model and deriving a low-cost, training-free method for measuring semantic bias directly from corpus statistics.
We develop EigenNoise, a zero-data initialization method for word vectors that synthesizes representations from Zipf’s Law alone, demonstrating competitive performance to GloVe after fine-tuning without requiring any pre-training corpus.
We present an unsupervised algorithm for inducing semantic networks from Wiktionary’s crowd-sourced data, creating a WordNet-like resource more than 5x larger than Princeton WordNet with over 344,000 linked example sentences.
Learn how computers understand words through mathematical vectors, from simple counting methods to contextual embeddings that power modern NLP.