Grace’s Reviews > AI Engineering: Building Applications with Foundation Models > Status Update

Transformers w/attention mechanisms (weighted probabilities to calculate outputs). Parallel inputs / sequential outputs (limited in context).

Embeddings (numerical rep. data) -> query . key vectors -> multi-layer perceptron (neural network that models inputs to nonlinear functions, models loss (compared w/correct output), and back-propagates (based on learning from correct output, change weight of model)