As discussed, a responsible and trustworthy AI practice must include interdisciplinary, diverse, and inclusive teams with different types of expertise (both technical and subject matter specific, including user or public-focused).

To provide the AI team with the best tools for success, the principles of DEIA should be at the forefront of any technology project. Responsibility for ensuring responsible design decisions that result in equitable outcomes falls on all team members, from the practitioners to managers.

The importance of this can be illustrated by examining all of the different decisions within the AI development lifecycle. These decisions, especially onesthat might be considered purely “technical,” require thorough consideration including an assessment of their impact on any outcomes. The following two examples illustrate the importance of DEIA considerations within AI system design and development:

1. How to handle missing values in the training data? It is well known that data sets used for AI often lack diverse representation or are overrepresented by certain demographics, which can lead to inequitable outcomes. One option is to filter the dataset to ensure it is more representative, but this could require disregarding some data which could reduce the overall dataset quality. An alternative is to collect new data targeting missing groups, but this comes with risks as well. How will you get the data? Will you pay for participation in the data collection in an ethical way? Are you now collecting even more sensitive data and if so what additional protections are needed?

2. Which metrics will be used to measure a model’s performance? What an AI system uses to determine it actually is working is an essential decision. Diverse and inclusive AI teams could be better positioned to suggest metrics that will result in equitable outcomes and processes that are more broadly accessible. An example of metric selection gone wrong occurred when a health insurance company decided to target the most in-need patients with the goal of providing more coordinated care to both improve their health and reduce costs. Unfortunately, the metric used was “who spent the most on their health care” which, because white people are more likely to use the health care system, resulted in underestimating the health care needs of the sickest Black patients. Even using a “seemingly race-blind metric” can lead to biased outcomes.

One suggested action to encourage deep consideration of key technical questions during AI system design is to implement iterative review mechanisms, particularly to monitor for bias, and be transparent regarding tradeoffs made in decision making regarding model performance and bias. This process begins with the assumption that there are biases baked into the model, and the review’s purpose is to uncover and reduce these model and historical biases.

These may seem like technical questions that a leader, program, or project manager may not normally focus on. However, successfully managing an AI project means establishing structures that ensure responsible and trustworthy AI practices are the responsibility of the entire team, and not just left to the day-to-day developers. As demonstrated, seemingly simple day-to-day design decisions that AI teams make have implications for marginalized communities. Contributors from the entire team, which can include designers, developers, data scientists, data engineers, machine learning engineers, product owners, and project and program managers must work together to inform these decisions.