What happens when you put your hand in the fire? There are two ways to find out:
My motivation with these blogs is to help you with Option #2 in the context of real-world Data & AI fires.
A little about myself: I am a Ph.D. in AI/Expert Systems with over two decades of experience building software systems, data, and AI. Over my career, I have built numerous Data+AI products and platforms, have 40+ issued patents, an O’Reilly book author…
87% of ML projects fail today!
These numbers should be taken with a grain of salt. Irrespective of the actual number, it does reflect reality — I have seen a significant percentage of ML-based projects never get into production!
The goal of this blog is to share my experiences on things that can go wrong in an ML project (they added up to 98!). The motivation with this post is for you to potentially avoid these landmines in your role as a data engineer, data scientist, ML engineer, data-business leader driving an ML initiative.
Every problem is not suitable for AI! Problem framing is important to ensure success in designing and deploying an AI product. In developing AI products, the following is my checklist to ensure we are solving the “right” problem.
Verified there is quantifiable business value in solving the problem
To deliver a well-tuned robust ML model deployed in production can range from 6–24 months depending on the complexity. Being crystal clear about the strategic value of the project is critical.
Verified that simpler alternatives (such as hand-crafted heuristics) are not sufficient to address the problem
Simpler alternatives are often overlooked. Also…
85% of AI projects never get to deployment! From the projects that get deployed, only 60% ever deliver on their promised impact! In a typical AI team today, not everyone is at the same level of experience required for building successful AI products. Team members vary in their experience with statistics, model foundations, data wrangling, operations, deployment, and product design thinking.
The key to ML is the availability of “right” data. “Right” data is a combination of right features/metrics, right distribution (IID) in the raw data, and the right labeling of the data samples.
The need for labeled data is dependent on the type of ML algorithm i.e., supervised learning requires labeled samples for training models. In 2020, the image/ video segment accounted for over 35% of the global revenue for data collection and revenue. …
AI teams invest a lot of rigor in defining new project guidelines. But the same is not true for killing existing projects. In the absence of clear guidelines, teams let infeasible projects drag on for months.
They put up a dog and pony show during project review meetings for fear of becoming the messengers of bad news. By streamlining the process to fail fast on infeasible projects, teams can significantly increase their overall success with AI initiatives.
AI projects are different from traditional software projects. They have a lot more unknowns: availability of right datasets, model training to meet required…
Data+AI is rapidly evolving with several rapid advancements in data, ML, AI technologies. In order to succeed, it is critical for teams to keep up with the new technologies as well as leverage experiences w.r.t. delivering faster business value. The best way for teams to learn-&-grow is Peer-2-Peer mentoring.
P2P mentoring is not a new concept. In my past experiences, I have applied it within software teams, data engineering, and data science teams. In this blog, I wanted to share details related to applying P2P mentoring within a product focussed AI team. …
Article originally published in VentureBeat
How do you start assembling an AI team? Well, hire unicorns who can understand the business problem, can translate it into the “right” AI building blocks, and can deliver on the implementation and production deployment. Sounds easy! Except that sightings of such unicorns are extremely rare. Even if you find a unicorn, chances are you won’t be able to afford it!
In my experience leading Data+AI products and platforms over the past two decades, a more effective strategy is to focus on recruiting solid performers who cumulatively support seven specific skill personas…