I'm building a suite of interactive tools (toys?) to help teach concepts from machine learning.
Lots of ideas from machine learning would be easy to grasp if only you could hold the data in your hand, poke it into the right shape, and see the model as it fits to the data in real time. While there are lots of great video resources for learning ML out there, as well as some excellent blogs, I haven't seen any comprehensive interactive tool like this.
I've been working on this interactive ML teaching tool in my spare time for a couple months now. So far, I've built large sections of the basic framework, but creating a robust tool takes time... And so I wondered — could AI help build a tool to teach AI?
In June 2024 Anthropic introduced their Claude 3.5 Sonnet model. A key new piece of the Sonnet model is Artifacts. Artifacts are pieces of code, text documents, or websites that appear in a separate window alongside Claude's main chat flow. If an artifact is dynamic, then it actively runs inside this side window. Let's ask Claude to build this AI interactive tool!
A First Failure
When I teach machine learning, I find myself over and over again drawing imaginary data on a grid, a la:
Let's start by asking Claude to generate and plot these data distribution pictures for us:
Here's what Claude came up with:
First of all WOW! Just a paragraph verbal description and out comes functioning code that (almost) does what I asked:
✔️ Visual display of synthetic data
✔️ Between 1 and 4 classes
You can press the "Generate New Dataset" to get new data points from a new collection of classes
✔️ Each datapoint consists of 2 features - one horizontal and one vertical
❌ Each class is a multivariate gaussian
In Claude's artifact the vertical axis is gaussian but the horizontal axis is not. Instead the horizontal axis is distributed uniformly
✔️ Random mean and random standard deviation for each class
My only other complaint is that this little tool is ugly. The color choice is html default. And what's with the 15 digits after the decimal point along the x-axis? In fact, the numbers along the x and y axes are meaningless and should simply be removed entirely. But I didn't give Claude instructions for the graphic design of the tool. For now, I'll assume I can fix the looks later, and focus on getting Claude to fix it's one major technical mistake:
What an agreeable AI! It (seems to) recognize its error and will fix it!
Err, nope -- exactly the same! I tried asking Claude to fix the issue a second time, but Mr. Agreeable did not seem to be able to rethink the code enough to actually effect a change.
So I started over, closing this conversation and opening an entirely new one.
Try no. 2
Maybe my first instructions were too complicated. Let's break down the request step-by-step:
Hey! It worked.
Well, alright. it didn't work the first time. Claude's first artifact was simply blank.
After pointing this out to Claude, the next artifact looked like this hairy mess:
But actually Claude now is generating a sample from a 2D gaussian -- except the AI is connecting the points with lines. So I asked for a scatter plot -- and then it worked.
Clearly, Claude is both very clever and very dumb. No human at all familiar with statistics would plot a gaussian distribution while connecting the dots with lines. But it seems if you give the instructions carefully enough, Claude can actually do what you want. Again, Wow!
Let's keep building this tool up slowly. Step by step...
1. A button to generate new samples, each sample with a different mean and covariance
2. Multiple classes
3. A slider to control the number of data points in each sample
Each of these worked like a charm on the first try!
And, Hooray! We're back to where we were during the first stab -- except now we actually have 2-dimensional gaussians!
Extending Further
The last technical flourish I wanted to add to data generation was a bit trickier. To help see why the datapoints end up where they do, I asked Claude to draw a 90% confidence ellipse around each multivariate gaussian. A 90% confidence ellipse is a contour line on the gaussian's 3d "bell" curve such that 90% of the time you when you pick a point from the distribution, the point lies inside the ellipse.
Drawing a 90% confidence ellipse on a multivariate gaussian is a wee bit technical*. So, could Claude?
You might think that an AI would be great at technical topics -- since computers are great at math and the AI lives in a computer. But in reality LLM's often struggel with math, and this was no exception. Claude's first try went like this:
*Specifically you find the transpose of the inverse of the lower-triangular part of the Cholesky decomposition of the inverse of the distribution's covariance matrix. And then multiply that matrix by a matrix of points that lie on a circle (or n-sphere) of radius equal to the value at which a chi-squared distribution is equal to 0.9, where the chi-squared distribution has degrees of freedom equal to the dimension of the multi-variate gaussian. (I figured this out by referencing this very helpful Math Stack Exchange page.
Claude did draw 90% confidence ellipses* -- just not the ellipses for the distributions that the data points come from! 😀
Fortunately, this time when I pointed out the bug, Claude figured out the solution after just a single try!
*Nice looking ovals! it's actually quite difficult to make each dash along the outline of an ellipse be equal length. I assume drawing dashed ellipses is built into the libraries that Claude is using to draw to the screen.
Up Next?
Well, it's not pretty. Too many numbers, hideous coloring and poor button layout -- but it does work.
Next time we'll see if Claude can continue to build this tool, and compare Claud's efforts to mine.
Comentarios