Renowned South East Asian University

Online learning platforms, universities and other learning centers need a way to automatically generate assignments/exam questions from textbook. This can help instructors efficiently generate assessments and provide a more engaging learning experience for students.

We developed a system to fine-tune a large language model (LLM) to generate relevant question-answer pairs from a given text source, such as a university textbook PDF.

• Data Preparation: The textbook PDF content was extracted, cleaned, and segmented into chunks of approximately 2000 words each. We tasked ChatGPT to generate question-answer pairs from these chunks, resulting in a dataset of 1800 pairs.
• Model Selection: After trying smaller models like Google T5 and GPT-2, we settled on the TinyLlama model with 1.1 billion parameters, as it showed better performance for our use case.
• Training Workflow: TinyLlama was fine-tuned on the question-answer dataset for 10 epochs, with checkpointing to capture model states at different stages. This aimed to improve the model's understanding of the textbook content and its ability to generate accurate question-answer pairs.
• Inference: The fine-tuned model can generate responses to text prompts (e.g., textbook sections) by formatting the input, specifying generation parameters, and decoding the output to obtain the final question-answer pairs.

TinyLlama Model at Huggingface

1. Efficient Assessment Creation: Instructors can automatically generate relevant assessment questions from textbook content, saving time and effort.

2. Improved Learning Experience: Students receive customized and engaging assessments tailored to the course material, enhancing their learning outcomes.

3. Scalability: The automated system can scale to handle large volumes of textbook content and generate assessments for numerous courses and subjects.

4. Continuous Improvement: The model can be further fine-tuned on additional data, improving its performance over time.

By leveraging large language models and fine-tuning techniques, this solution streamlines the assessment creation process for online learning platforms, ultimately benefiting both instructors and students.