How we design Machine Learning Models


One of my favorite questions I ask customers on the topic of machine learning is to list out the top 3 words that come to their mind on Forecasting vs Predictive.

Here are the most common answers

Forecasting: Business need, Discipline and Math based.

Predictive: No logic, black box and complex.

Hope I got your attention by now.

In today’s world there is a general misconception that everyone can start using machine learning (ML) based predictive analytics right away by skipping the basics. We wanted our product to have some key characteristics that helps customers embrace ML step by step. Listed below are few guidelines that helps us to stay to the core.

Design Criteria: User as an active participant.

One of the big problems in predictive analytics is either users take it very seriously or ignore completely. We want the user to be an active participant and above all a critic. We want to provide a level engagement by bringing in past, present and future together. This provides a holistic view for the user to look at facts, range of possibilities and outliers without any wild guesses. E.g in our application we bringing the historical sales (past), current sales pipeline (present) and User judgement (future) together in one single view.

Design Criteria: Never mistake a clear view for a short distance.

With so much hype around AI and machine learning, the human talent and experience is taking a back seat. We strongly believe in machine learning applications that augments human judgement. It’s easy for models to take one piece of seeming strong information and have results skewed based on it.

Machine learning will change the way sales teams make forecasts, but it’s important to realize that not all models are built the same way. Consider these two scenarios:

  1. Model A looks at current data and determines the attributes of success. For example, an algorithm identifies that when a ZIP code is present for an opportunity, the chance of winning the deal is high.
  2. Model B looks at historical snapshots and determines the attributes of success. For example, an algorithm identifies that the close date of an opportunity has been pushed by more than 5 times in the last few days.

Which opportunity do you think is more at risk?

Models need human judgement to find the difference between improbable and impossible events.

Design criteria: It’s not a one time event

It’s easy to analyse data find insights make predictions and move on with our daily routine. Forecasting is not an one time event and is never accurate if we approach that way. It has be often, regular and continuous learning. We want to provide an user experience that captures change and uncover hidden trends that may be difficult to notice just by looking current data alone. User sentiments change, things changes at a company level both inside and outside. We want to provide a systematic and disciplined approach that give details on what changes, insights on why the models is predicting certain outcome – we want to make this human – machine interaction a dialogue and not an one time event.

We measure our quality of our results not by %accuracy of prediction but by helping users changing the behavior. After all predictable results is because of disciplined journey and there is no shortcut to it.