Frequently Asked Questions

Everything you need to know about RAG and the Virtual TA project.

RAG Fundamentals

Understanding Retrieval-Augmented Generation.

Project: Virtual TA

Details specific to the TDS Virtual Teaching Assistant project.

Data Collection & Processing

How to gather and prepare data for your Virtual TA.

Implementation Details

Technical aspects of building your Virtual TA.

Hosting & Deployment

Information on deploying your API.

Testing with Promptfoo

Learn how to test your Virtual TA using Promptfoo.

Evaluation & Submission

Understanding the project evaluation and submission process.

Contents

Frequently Asked Questions

RAG Fundamentals

What is RAG?

How does RAG work?

What are embeddings?

What is a vector store?

RAG vs. Fine-tuning: What's the difference?

What are the advantages of RAG?

What are the limitations of RAG?

Project: Virtual TA

What is the main goal of the Virtual TA project?

How do I get started with the Virtual TA project?

When is the project submission deadline?

What data sources will the Virtual TA use?

What is the expected input for the API?

What is the expected output format for the API response?

Data Collection & Processing

How do I scrape data from Discourse?

How do I handle rate limiting when scraping Discourse?

What should I do if I get a 404 error on Discourse?

How should I process and store the scraped data?

Is there any pre-scraped data available for the project?

Implementation Details

Do I need to pay for OpenAI embeddings?

What are the key requirements for the API endpoint?

Which programming language should I use?

Can I use a managed RAG service instead of building it myself?

Which embedding model should I use?

Can I use Ollama for this project?

Hosting & Deployment

Where can I host the Virtual TA API?

Does my API need to be available 24/7?

Can I host my API locally?

What if I use ngrok for hosting?

Testing with Promptfoo

How do I set up Promptfoo for testing?

How do I configure Promptfoo to use my API endpoint?

How can I add custom assertions to test my API?

How do I add more test cases?

How do I use LLM rubric evaluation with Promptfoo?

How do I get an AI Proxy token for Promptfoo testing?

Can I see a complete Promptfoo configuration example?

How do I troubleshoot common Promptfoo errors?

How can I automate Promptfoo testing?

Evaluation & Submission

How will the application be evaluated?

What are the pre-requisites for my project to be eligible for evaluation?

How can I self-evaluate my application?

What are the bonus marks for?

What do I need to submit?