I love Quantization, you can see some of the models I have Quantization on my Ollama or Huggingface profile. What Is Quantization? In machine learning, quantization is a technique that reduces the computational and memory costs of running a model by lowering the precision of its numbers. Instead of storing
Setting the Stage: From LSTMs to Transformers When researchers introduced sequence models like the long‑short‑term memory (LSTM), they were responding to the problem that recurrent neural networks had trouble remembering things from far in the past. LSTMs introduced a gating mechanism that let the model decide what to
The original attention mechanism and its scaling cost Self‑attention, introduced around 2017, allows each token in a sequence to interact with every other token. The mechanism builds a dense attention matrix whose size grows quadratically with the sequence length. This design is elegant but expensive: the time and memory
Word vs Sentence Encoders — Digging Deeper In machine learning, encoders convert raw text to vector embeddings that capture semantics. Historically, word-level encoders like Word2Vec and GloVe assign each token a single static vector. This is easy to train and interpret but it cannot differentiate the same word in different
News
This is DeepNeuro.AI Research Notes, a brand new site by Richard Young that's just getting started. Things will be up and running here shortly, but you can subscribe in the meantime if you'd like to stay up to date and receive emails when new content