Where Tbilisi's AI CommunityConnects & Innovates

Agenda:

• Why do we need Embeddings in LLM era?
• Context Length and RAG as Application
• Why LLM can't be used for Embeddings?
• Causal Attention, Next Token Generation vs Contrastive Learning
• Sparse vs Dense Retrieval vs Hybrid (reciprocal rank fusion)
• Sparse TF-IDF (LSA & PCA over TF-IDF, etc…)
• SPLADE (query expansion)
• Bi-Encoders / Cross-Encoders / Late Interaction
• Cross Encoders have attention
• Bi Encoders are good for retrieval (LHS, HSNW, …)
• Latent Interaction - kinda mix of both worlds
• Contrastive Learning, Triple Loss, Siamese Networks
• How to choose right model
• Speed & Size VS accuracy
• Domain? Code vs Text for example (Tokenisation Difference)
• SFT, domain adaptation
• BEIR, MTEB benchmark