In many organisations, the path to an AI-enablement is littered with mistakes, false starts and wasted investment. In an environment where 70% of digital transformations fail, it is crucial to maximise the likelihood of success by learning from others who have gone through this journey. 

Ahead of the 4th annual AI For Business Summit, we spoke with four industry experts about the challenges involved in scaling AI. Preparation is everything. We asked four experts to share the top tasks they believe should be completed before attempting to scale AI.

1. Understand your data quality

Dr Ian Opperman, the NSW Government’s Chief Data Scientist working within the Department of Customer Service, says that it’s crucial to understand your data quality and limitations before attempting to scale AI. “Poor quality, biased or incomplete data will limit its possible uses and the level to which you can rely on your insights and system decisions.”

Joanna Gurry, Chief Data Officer and Executive General Manager, NBN Co, also stresses the importance of good data. “Get a peer review or independent checks done on the code, the models, and the quality of the data you are using. A small error will become a big problem once you start scaling”, she says.

2. Anticipate hurdles

“Whilst we’re all limited by our unknown-unknowns (unforeseeable conditions), it is worth thinking through the types of problems you might reasonably address as systems scale before investment starts”, says Oppermann. “This will help set expectations and identify gaps”. 

More specifically, Gurry says its vital to have confidence that your systems and processes can scale upstream and downstream from the AI component; otherwise you’ll experience bottlenecks.

3. Build a balanced team

“Scaling AI requires more data scientists, but it also requires business analysts and data engineers to support them”, advises Gurry. “Hiring more data scientists without the additional skills will slow the team down and frustrate everyone. Having good balance in hiring and types of experience has helped overcome the challenge with growing the team despite a tight market.”

4. Re-evaluate your data governance needs

According to Oppermann, most problems that occur in AI scale-ups stem from increased access by more people to a wider range of data and insights. “Think this through and ensure data governance is part of the scale-up plan”, he says.

(l to r) Dr Ian Opperman, NSW Government’s Chief Data Scientist; Joanna Gurry, CDO and Executive GM, NBN Co; Latitude Financial Services’ CDO, Kshira Saagar; Angela Kim, Head of Analytics and AI for the Teachers Health Fund.

5. Remove bias

Angela Kim, Head of Analytics and AI for the Teachers Health Fund, says it’s vital to embed an ethical, explainable, diverse and inclusive AI culture in the organisation to avoid unexpected consequences such as AI bias.

“In many cases, an AI pilot involves automation or an augmented process that is based on AI algorithms. Hence if there is a bias in the dataset or the way data has been collected, it naturally filters through the algorithm that is being created, leading to a multiplier effect of amplified bias which will be served on the product or services with unfair treatment to particular minority cohorts of customers.”

An example of this in practice occurred in 2018 when Amazon’s recruiting AI was found to be biased towards women due to the data it had been fed. 

6. Measure, monitor, message

Latitude Financial Services’ Chief Data Officer, Kshira Saagar, says that a triple-M framework is useful when it comes to attempting to scale AI.

• Measure: “Use an existing, well-known KPI and define success as a series of bands around that KPI. For example, a personalisation algorithm will improve time spent on site from 1.5 minutes to at least 2.5 minutes or more; if it increases to just 2 minutes, you should spend another 6 months working on it.”

• Monitor: “After setting your KPIs, ensure performance is monitored transparently and widely. Be ready to act swiftly and decisively if things don’t work out as expected.”

• Message: “Once the algorithm works, communicate its value in a humane, understandable, non-technical way to all the people who will be impacted positively or negatively because of it.”

Focus on buy-in

According to all four interviewees, the most challenging aspect of scaling AI is not to do with technology, but with people.

First, do your homework in terms of stakeholder needs and build a solid use case well before any investment begins. “You should be very certain that the use case is valid, and the business really, truly wants the solution”, says Gurry.

Saagar stresses the importance of building buy-in from the employees, partners and clients who will be most impacted by it. “AI algorithms come with a bad PR image”, he says. “People think they’re going to replace jobs, and in extreme cases think AI is here to destroy humanity (thanks to the Terminator films). It’s paramount, therefore, that the AI is explainable and that the people building it are willing to be told they are ‘not as smart as’ the person currently doing the same job. Because let’s say it; humans are still very smart compared to some of the most complex algorithms out there.”

Kim also recommends focusing on developing team buy-in. “The goal is to change employees’ attitude from perceiving AI as something that will replace their work, to seeing it as a collaborative tool that knowledge workers need to embrace in order to automate mundane and repetitive tasks”, she says. “Aim to convert the culture of the business into ‘test and learn, fail fast and experiment often’ to take advantage of AI’s amazing capabilities.” 

In Kim’s experience, success will come once the culture and genuine partnership are established. “After that, knowledge workers will acknowledge that AI can free up their valuable time and enable them to focus on value-added cognitive work that technology cannot replace.”  

Manage expectations

Unlike a simple proof-of-concept project which can get people excited and can show amazing results fast with little expense, a scale-up will take much more time, more money, and produces far fewer exciting results per dollar. Oppermann says that managing major stakeholder expectations is therefore critical. “Doing this through a multi-horizon plan works well where increasing investment and time is rewarded by scaled up capability and value.” 

Saagar agrees that a scale-up isn’t about quick wins. “Tie all your AI initiatives to a tangible outcome/value to the business and your customers”, he says. “It’s not about the coolest or the ripest technology or other bragging rights, but building something that will make lives easier every day.”

Advice for those just starting on this journey

Plan ahead and expect the unexpected

“Spend more time than you think is necessary thinking through what you are trying to do, why you are trying to do it, and plan ahead for when things go wrong”, says Opperman. “Think about how data can be used to bring different stages of the project to life.”

Kim agrees that to scale AI, stay competitive, meet customer expectations and deliver best outcomes, analytics leaders need to get the foundation right at the beginning. “They need to learn how to match models and techniques to their specific problems and challenges, because there is no one-size-fits-all solution. Tailor your plan to align with the organisation’s vision and purpose.”

Gurry warns that paths to production are not always the same. “They can be dependent on the systems you are introducing the models to.  This means some of the learnings and design patterns are not always repeatable. Working very closely with the business teams to integrate changes to the processes that will use the model, and planning the changes closely with IT have been important to get models into production and not just in ‘insight’ mode.”

Focus on security and privacy

Kim recommends following the ART principle – Accountability, Responsibility and Transparency in AI – to embed security and privacy in the design phase. “This will help the organisation develop a clear understanding of the limitations of the AI tools they are using”, she says.

Scale up in stages

Take an iterative approach. “Do it in phases”, says Oppermann. “Try, test and learn, then scale to the next level and repeat as needed.”

Just do it

“There is never a perfect time to scale up, and never enough money,” says Gurry. “Just get started but be sure to make every move in strong partnership with the business”.

THE AI FOR BUSINESS SUMMIT

Now in its 4th year, the AI for Business Summit 2021 will explore how to strategically scale AI and investments to grow revenue through innovation, improving services, reducing risks and making sound decisions backed up by data.

Join us at the AI for Business Summit 2021 where we'll provide you with a melting pot of case studies, discussions, critical learnings, and future projections focused on the role AI will continue to play in Australia's economic future.

View the agenda here. Register here. 

https://hubs.la/H0PNlH30