We listen in on a fascinating discussion within SAP about the increasing role of Artificial Intelligence in our society and the growing importance of having an ethical framework to understand its consequences.

We’ve talked with SAP a couple of times before. The company has built itself a proud reputation as the global leader in business applications and analytics software, as well as a market leader in digital technologies. SAP is the world’s largest enterprise cloud company with over 195 million cloud users, and over 437,000 customers in more than 180 countries ranging from small companies to global organizations.

78% of the world’s food and 82% of the world’s medical devices are distributed through SAP customers, while over 78% of all business transactions around the globe come into contact with SAP software.

In other words, SAP is a company that is uniquely positioned to talk about the benefits of the coming Fourth Industrial Revolution, as well as the risks that accompany them. This is particularly true for Artificial Intelligence. We talked with two of SAP’s global thought leaders, discussing how humanity’s approach to AI has to grapple with not just the practical challenges of the technology still very much in its infancy, but the ethical conundrums that arise as a result, as well.

“Artificial Intelligence is the new electricity,” explains Rudeon Snell, the Senior Director of Intelligent Enterprise Solutions for SAP EMEA. “That’s the transformative influence AI is set to have on our global societies. With the accelerated pace of AI development, it has increasingly led to a huge interest in ethical AI and how, as imperfect human beings, we are teaching AI the differences between right and wrong.”

Snell believes that one the key challenges with AI systems is that as they continue to evolve and become more effective, humanity will place increasing levels of trust in them when it comes to decision making, especially as these AI systems transition from being perceived as mere tools in science laboratories, to operating as autonomous agents making autonomous decisions, driving specific business outcomes.

Robotics imbued with Artificial Intelligence is already capable of fulfilling a wide range of tasks including performing surgery, driving vehicles itself, augmenting customer experiences and performing its own financial transactions. But with every new application come new ethical and legal questions that Snell believes we need to establish global guidelines for.

“Think about the ethical concerns around the potential human rights violations affected by issues of privacy, for example, mandatory facial recognition, the challenges posed by legal liability in the case of an autonomous vehicle malfunction causing potentially fatal accidents, or the risks posed by weaponised autonomous AI,” Snell points out. “What guidelines will be used to in

form how AI value human life? How will determinations be made by AI, when it has to weigh the value of one human life over another? Who determines that valuation? And, as importantly, what about factors that are not expressed with data, which is the very foundation of AI, such as the value of another person, the value of emotional connections, the value of a relationship? The two biggest challenges with data unfortunately, is that data quite simply just does not data give you ethics and data is often laced with bias.”

This last point is absolutely crucial. While science fiction stories routinely return to the plot of “Artificial intelligence gaining self-awareness and going against their programming”, the far greater danger is that the programming itself will be imbued with all the biases and preconceptions of their programmers, but these preconceptions will become invisible because it’s “What the computer said.”

Opening Pandora’s Box

This isn’t an area that’s being neglected. There are many researchers grappling with the topic of ethics in AI, debating the aspects of what we use AI for, who has access to the technology and how Artificial Intelligence can share human values when “human values” is something that differs wildly between different geographies and cultures. Snell firmly believes that these are challenges that it is the responsibility of technology companies to address.

“With the tremendous opportunity AI is set to usher in, the trillion-dollar markets it is set to unlock, we as technology providers have an equally tremendous responsibility to ensure we contribute meaningfully to the creation of ethical frameworks, regulatory oversight and legal guidelines that can help mitigate the potential misuse of AI and lead to safer more inclusive adoption of AI,” Snell says. “A driving force steering how we, as SAP, come at this notion of ethical AI,

is embodied in how we ensure, that as custodians of AI-powered industry solutions, we are able to contribute meaningfully to the countries around the globe we operate in, making a positive difference to the lives of the citizens of those countries, so that the benefits that we are providing to both public and private institutions, can be enjoyed by all.”

David Judge, Vice President for SAP Intelligent Enterprise Solutions, agrees with Snell on the tremendous potential Artificial Intelligence can have, as well as the potential pitfalls involved.

“AI technologies hold tremendous promise to improve organizations, customer experiences, and the lives of every human,” Judge says. “It also holds tremendous pitfalls, both mundane (a poor customer experience) and dramatic (discriminatory action based on personal information). Aside from overt malicious intent, care is necessary to ensure that unintended problems aren’t embedded, automated and amplified. For example, data bias can negatively impact AI software, and in turn, individuals and customers. This is particularly the case when there is a risk of causing discrimination or unjustly impacting certain groups.”

For SAP these questions are not hypothetical, these are practical challenges that SAP needs to address in the immediate present and the near future. Their position as a market leader in enterprise technology, serving over 400,000 customers, demands that care be given to these issues. The work SAP does, after all, influences the lives of billions on a daily basis.

“Our identity and purpose are grounded in the principle of helping the world run better and improving people’s lives,” Snell points out. “As such, one of our key focus areas for ethical AI is to ensure a trustworthy, consistent, high-integrity, high-quality data core is the foundational pillar, that enables the Intelligent Enterprise as the vehicle for this transformative purpose.”

This means that SAP considers the ethical use of data and AI an absolute core value, which is why SAP was the first European technology company to announce its guiding principles for AI development, in addition to the creation of an external AI ethics advisory panel.
A core belief in assembling ethical AI guidelines and principles, Snell explains, is understanding the use and impact of data.

“AI systems feed off data,” he says. “If AI is the new electricity, data is the grid that it runs on. AI systems look at data, evaluate that data against its goals and then find the most optimal path towards achieving those goals. Data is absolutely critical for AI systems to be effective and to drive true business value. This is evidenced in AI’s ability to make decisions based on data and with decisions, there are trade-offs that must be taken into account.”

Rules of Robotics

Snell tells us that it is imperative for technology companies to have a set of principles and guidelines that shape the development of AI and its use, so as to ensure the adoption and inclusion of AI from the first world to the third world.

“We must ensure we hold true to our common values and beliefs that guide our decision making on AI,” Snell says. This is why SAP’s guiding AI principles form the foundation upon which AI within the Intelligent Enterprise is manifested.”

These principles need to recognise the significant impact AI has on people as a whole, ranging from SAP’s own customers to wider society. The business has designed their guiding principles with the goal of steering not just how their AI software is deployed, but how it is designed and developed in the first place, ensuring that it helps people’s lives improve.

The first of these principles is “Being Driven by Values”, which means that the company recognizes how AI, like any technology, has the scope to be used in ways that are not aligned with SAP’s intentions, principles or operational guidelines. While developing AI-based software, SAP is committed to remaining true to their Human Rights Commitment Statement, as well as the UN Guiding Principles on Business and Human Rights, the law, and internationally accepted norms. If necessary, SAP’s AI Ethics Steering Committee serves to provide SAP’s teams with advice on how the company’s principles apply to specific use cases. The development of an AI product comes into conflict with SAP’s principles, they will work to prevent the inappropriate use of their technology.

Their second principle, “We design for people” may sound like it is stating the obvious, but recent history has shown that as programs become more complex it is all too easy for processes to shift to suit the algorithm, rather than the people who use it. SAP has set itself the goal of creating AI software systems that are inclusive and that seek to empower and augment the talents of their diverse userbase. SAP offers human-centred user experiences through augmentative and intuitive technologies and leverages AI to support people in maximising their potential. This means designing systems in close collaboration with users in a multidisciplinary, and demographically diverse environment.

The third principle is to “Enable businesses beyond bias”. We’ve already discussed the ways that Artificial Intelligence can enshrine its creators’ biases and preconceptions in code, making them harder to detect while enforcing them more strongly. This is an even greater concern when there is the risk of causing discrimination or of unjustly impacting underrepresented groups. To avoid this, SAP requires its technical teams to spend time learning about and gaining a deep understanding for the business problems they are trying to solve, as well as ensuring the data, the fuel the AI runs on, is of the quality this demands. Because a programmer’s own unconscious biases can affect how an Artificial Intelligence processes that data, SAP works to build diverse and interdisciplinary teams, and is investigating new technical methods for mitigating biases.

SAP is also committed to supporting its customers in leveraging AI to build products that can help their businesses move beyond bias.

The fourth principle is to “Strive for transparency and integrity in all that we do”, which is the sort of wording you might find in any company’s “Principles and values” document, but these words take on special meaning in a technology environment, especially where all-too-often AI can be treated as a magic black box we cannot see inside.

SAP’s systems are held to specific standards that take into account the technical ability and intended use of those systems. In a market where vague promises and mysteriously opaque

processes can be the norm, SAP clearly communicates to their customers what a system’s input, capabilities, intended purpose, and limitations will be. Part of this means offering the means for customers and users to oversee and control those systems themselves. A common fear when it comes to implementing Artificial Intelligence is that it takes control out of people’s hands, and SAP’s products are developed to ensure that never happens.

It is on a related note that SAP’s fifth principle is to uphold quality and safety standards. SAP’s AI software won’t reach its customers until it has gone through a thorough quality assurance process, which is continuously adapted as necessary. This process means testing under real-world conditions, ensuring products are fit for purpose and meet specifications. The testing process is done in close collaboration with users and customers so that the end product has unquestionable quality, safety, reliability, and security.

The sixth principle comes back to Snell’s point about the critical role data plays in AI systems. Data protection and privacy are in-built requirements at the core of all of SAP’s products and services. Just as SAP’s processes are transparent, so is the way they use data, and the company communicates clearly the how, why, where and when of the customer and anonymised user data is used in SAP’s software. SAP goes beyond legal and regulatory requirements to research the next generation of privacy-enhancing methodologies and technological solutions with leading global academic institutions.

Finally, and perhaps most important, SAP believes in engaging with the “Wider societal challenges of AI”. While all of SAP’s other ethical values regarding AI concern areas that are within the company’s control, we are talking about one of the most rapidly developing areas of technology and challenges that are emerging beyond what we have currently been able to foresee.

Preparing for the Unknown

AI is a sector where developers need not only address current technological, economic and societal problems but also need to bear in mind problems, circumstances and even technologies that do not yet exist. They need to understand the economic impact products might have, such as how society and industry can work together to make sure workers and students are ready for an AI economy in an AI-first world. Even issues as huge and fundamental as economic redistribution, social safety and economic development must be accounted for. On a social level, seemingly etheric concepts such as the value and meaning of work, and how automation can impact that, need to be considered and understood. For instance, there is a lot of development right now going into AI providing personalised experiences for customers. But what about the concerns we see in data privacy, data sovereignty and data ownership? How will this affect the vulnerable people who rely on those products and services for a living? How do we ensure we help protect the most vulnerable to ensure the gap between the haves and the have nots, does not exponentially increase? How do we help protect against the creation of an entirely new techno-classist economy?

SAP believes that to prepare for coming developments in AI, it is essential to break taboos and be able to confront issues of safety and security, as well as other ethical dilemmas that arise from new applications of AI. SAP considers its role to be that of an active voice in the discussion of AI, particularly by engaging the company’s AI Ethics Advisory Panel.

But Snell emphasises that the important thing to remember is that rolling out ethical guidelines for AI is an ongoing process. The seven principles SAP has laid down aren’t the end goal, but a first step in the journey to managing the development and application of new AI products.
“It should create awareness and encourage employees to take the topic seriously,” Snell says. “Alongside the broad spectrum of massive open online courses on offer about AI, SAP has designed an internal machine learning education program for employees that will also deal with the topic of AI. In addition, there will be opportunities for SAP employees to become more closely involved in the topic including one concept of having Employee Ambassadors, particularly from teams that are working closely with AI.”
Unlike other industries, even in the technology sector, businesses working in the Artificial Intelligence sector have obligations that go beyond simply legal and regulatory compliance. The fact is, the technology may bring up ethical issues before there is a legal framework to guide or rule on such issues.
“The world needs technological innovation and breakthrough to drive growth. The pace at which technology continues to develop means regulation often plays catchup and struggles to keep pace with how technology is impacting society,” Snell says. “The ubiquity of cameras, for example, has had many benefits, such as seeing the truth in real-time, as is the case with police body-worn cameras. But how is this contrasted with an individual’s right to privacy? Or in the case of autonomous vehicles, who will decide how split-second life and death decision will get made? How is that decision calculated?”

Judge adds, “Though the pace of change is rapid and accelerating, it is more important than ever to institute ethical frameworks, and guiding principles which are broad enough to be resilient to unforeseen developments. As with the cloud revolution, useful definitions and categories only began to emerge as the market developed, however, good management principles remained relevant. The continued advancement of AI technologies will shine a bright light on the organisations that have effective governance around ethics, and those that do not. Our own guiding principles are a commitment to move beyond what is legally required and to begin a deep and continuous engagement with the wider ethical and socioeconomic challenges of the AI revolution.”

Snell and Judge agree that the industry needs to show maturity and come together to define the way forward concerning ethical AI. They point to the Cybersecurity Tech Accord as an example of the sort of thinking that will be necessary to deal with the ramifications of AI. Snell calls for the creation of a “new Geneva convention” for today’s constantly evolving geopolitical, AI-first world.

“For us, as SAP, our AI principles and guidelines are a commitment to move beyond what is legally required and to begin a deep and continuous engagement with the wider ethical and socioeconomic challenges of AI,” Snell says. “We are constantly looking at the ethical and societal implications of the latest advances in technology and contributing to the public debate about this subject. The ethical guidelines for AI are used to address concerns about AI and aim to ensure that SAP’s AI portfolio maintains the integrity and continued trust in our company and our solutions.”

It is a process of thinking and technological development but it is also a process of conversation, with customers, partners, employees, legislative bodies and civil society-at-large. The goal of that conversation is making SAP’s guiding principles an evolving reflection of that discussion and the constantly evolving technological landscape. It is a challenge that ranks up there with some of the greatest technological tasks humanity has ever taken on. But more than that, it also serves as a foundation that we will build on to address future challenges.

“The new Manhattan Project is AI safety,” Snell says. “After all, it will be human choices that define the future of AI. As global citizens and stakeholders, we all need to do our part to ensure our continued prosperity, as this technology helps us tackle and conquer humanity’s grand challenges.”

David Judge, Vice-President for SAP Intelligent Enterprise Solutions

Rudeon Snell, Senior Director of Intelligent Enterprise Solutions for SAP EMEA