How to report the progress of your AI project?

FAQs for AI Zero → One (Episode #7)

Transcript

Hey, everyone. Welcome to episode number seven, where we cover FAQs in going from zero to one in AI. Today’s question is how to report progress on your AI project. Now, the reason this question comes up is traditionally in software, or software engineering, we’ve got really good in how we define, break down the problem, and execute. And you typically have software [00:00:30] projects that go more or less in a straight line, right? Where you have a well-defined set of tasks, and milestones defined.

When you are missing those milestones in a timely fashion, that’s where you use the red, yellow, and green status. The project is in red if it has completely lost track. Yellow, potentially losing track of what was envisioned versus what is happening. And green means on track, [00:01:00] will deliver, and so on.

Now, when you think about an AI project, this approach is n-, is, you know, not very sufficient. In fact, it’s useless. And, and the reason is within an AI project, you have a whole bunch of different unknowns. Firstly, you have the data, the dataset, and the data features. The other dimension is the model space, which model to use, traditional, deep learning. Again, [00:01:30] within deep learning itself, if you’re looking at neural nets, there is a very wide variety that can be applied.

The third dimension also is model tuning. Uh, be it really understanding what are the right hyperparameters, what’s the right configuration, uh, to be applied. And obviously, the last dimension is, you know, the operational steadiness. You can build it, you can build a prototype, but if your pipelines are unstable, you cannot [00:02:00] really launch the model and, and ensure that, uh, the model also has the right fairness, security, and so on.

So given these unknowns, if you think of the red, yellow, or green model, a green doesn’t tell you much, because the product just keeps vacillating between these. Instead, a better way to talk about the status of your AI project is to use a maturity scale. [00:02:30] So what a maturity scale means, it actually tells you…