To do - TAI Course

[louisfb01]

• Create GitHub repo: https://github.com/towardsai/ai-tutor-rag-system
• Create a paid account with OpenAI and get your API key.
  1. Add key to env variables
• 1- Create an initial script for answering questions with OpenAI GPT 3.5. Basic function that takes an input, prompts around, and answers a question.
  1. Code from scratch
• 2- Add super basic rag with numpy and cosine similarity and embeddings ada function and a small dataset example in json.
• 3- Then compare with llamaindex to introduce it and why we use it in the future
  1. Update function to do RAG with llamaindex. Basic example but with llamaindex.
• 4- Replace database from json to Chroma or other
  • • script to embed and create a vector store from csvs
  • • script of how we chunked and got the data from an example using useScrapper https://usescraper.com/
  • • with basic chunking script (nb char)
• 5-Improve prompting for the question and add sources (references)
• 6- Advanced: write a script to create questions for the dataset with GPT4
  1. Script for evaluation script ragas or other with llamaindex or other
     1. Run evaluation
• 7- Improve chunking (sections, titles in sections…)- re evaluate
• 8- Script for Fine-tuning embedded based GPT4 generated questions above.
• 9- Replace Ada with Cohere or a better embedding model or HF.- re evaluate
• 10- Add reranking or open-source model (Cohere?)- re evaluate
• 11- Add Hybrid search- re evaluate
• 12- Improve query (reformulation, more details…)- re evaluate
• 13- Add router for data source optimization - re evaluate
• 14- Add chat feature to go back and forth with the bot ([llamaindex](https://docs.llamaindex.ai/en/stable/module_guides/deploying/chat_engines/root.html))
• 15- Replace open ai with open source models - re evaluate
• 16- ??? Fine-tune open source model (but cannot be used commercially since dataset built using gpt 4)
• 17- GPT4 (or 3.5) "judge" for re-ranking.
• 18- Add visualization lessons? Based on https://itnext.io/visualize-your-rag-data-eda-for-retrieval-augmented-generation-0701ee98768f, and https://github.com/Renumics/rag-demo/blob/main/notebooks/visualize_rag_tutorial.ipynb
• 19- Write the lessons from code built (notebooks initially, then we need to teach to replicate our full repo without giving it to them. They need to learn and work!). Report to the full syllabus (https://www.notion.so/seldonia/Full-syllabus-564070f715b2455d9a6b945b0b470c6b). Don't forget to re-use parts of the ebook we just did to save time.
• 20- Add multi-lingual section with image showing embeddings are the same, so valuable to have content in different languages, etc...

[louisfb01]

cc @AlaFalaki @omar-sol