Making the First-Ever AI LSAT Coach

This January, I began work on Daphne, the first-ever AI LSAT coach. Blown away by the abilities of ChatGPT, I set out to try to use the incredible technology behind it to supercharge LSAT prep. Since then, I’ve spent close to 400 hours building, tinkering, and playing with GPT-3, then GPT-3.5, and finally GPT-4. With Daphne finally done and integrated into Apollo’s Logical Reasoning Course, here’s everything I learned in the process about the strengths, limitations, and potential future uses of educational AI:

Advantages of an AI Coach:

1. Availability: As a piece of software and not a human being, Daphne is available 24/7. Whether you’re caught up in a late-night study sesh or up early for some pre-class prep, an AI LSAT coach is always there, ready to assist.
   
   

2. Cost: In my opinion, this is the biggest advantage right now. Because you’re not paying for anyone’s time, AI LSAT coaches are many orders of magnitude cheaper than human coaches. Having (near) coach-level help at a self-study price could make a world of difference for lower-income students, and help democratize test prep like never before.
   
   That doesn’t mean there are no costs, though; software like Daphne does have much higher up-front costs, both in time and money. There are also per-message fees associated with using the OpenAI GPT API, although these fees are still much smaller than what you’d pay for a coach’s time.
   
   

3. Accessibility: For students with social difficulties, an AI coach can eliminate many of the difficulties of working with a human tutor. There’s no fear of embarrassment or awkwardness when chatting with Daphne.
   
   

4. Speed: Written communication is faster than verbal. In the time it might take a person to explain a single problem, an AI could break down an entire practice test. This can be a massive advantage for fast readers/learners.
   
   Written communication can also be quickly referenced later, without having to scroll through an hour+ long video recording.

Disadvantages of an AI Coach:

1. No video: While high-quality text-to-speech technology is already available, generating believable, (non-creepy) CGI video content to accompany it is still difficult and expensive. This won’t matter to auditory learners or those who learn best by reading, but could be a deal-breaker for visual learners.
   
   

2. The “Human Touch”: While GPT-4 is pretty good at faking empathy, AI still has a long way to go before it can provide the level of emotional support and encouragement that a human can. Even as AI improves, some students may feel uncomfortable just knowing that they’re talking to a bot and not a real human being. This is the only issue that may not go away even as AI improves.
   
   

3. Quality: Although AI-generated explanations are usually very good, they can occasionally miss the mark. GPT-4 actually took the LSAT and scored a 163, which is an impressive score, but still a ways off from the 177 average of Apollo’s coaches.

What’s Next:

Here are a couple of the features currently in development for Daphne:

1. Support for Logic Games and Reading Comprehension

2. Guided blind review

3. Text-to-speech

4. AI tech updates as new OpenAI models are released

But what about AI coaches in general? Does the development of generative AI spell the end of human tutoring?

Probably not. While AI coaching presents some amazing new possibilities, it still has some significant drawbacks that make human coaching preferable for many students for whom cost is not a primary concern. For now, the most powerful use case of AI coaches is as a complement to human coaches, bridging the gaps between coaching sessions and allowing students to spend their human coaching sessions more productively.

The place where this technology will likely have the most immediate impact is with more price-sensitive students who would otherwise be unable to afford coaching, who will now be able to get personalized support without paying for coaching. My guess is that AI LSAT assistants will become standard-issue in almost all self-study courses within a few years, both for the LSAT and everything else. For now, though, there’s only one place you can find it: Apollo’s Self-Study Course

Got questions about Daphne, Apollo, or the LSAT?