Re: Comment around fairness and ethical concerns around AI and global competitiveness
Dear Co-Chairs Delaney and Ferguson, and Commissioners:
The U.S. Chamber of Commerce's Technology Engagement Center ("C_TEC") appreciates the opportunity to submit feedback to the Commission on Competitiveness, Inclusion, and Innovation's second request for information on fairness and ethical concerns and global competitiveness issues surrounding artificial intelligence.
We appreciate the Commission's efforts to work with all stakeholders to identify durable bipartisan policy recommendations to effectively develop A.I. With the right regulatory framework, the United States can be the leader in using artificial intelligence help society and improve quality of life.
C_TEC comments only on specific questions and issues that the Commission has raised.
Regulatory Frameworks:
Regarding question four, "how should "high risk" be defined? What factors should be considered? Is it linked to human rights violations? Is it linked to the individual's health, safety, and fundamental rights?" A.I. has different applications and different impacts. We believe risk varies for each sector and that industry specific definitions of risk and risk tolerance are appropriate.
Regarding question eight, "Are there A.I. application risks that are more appropriately addressed by non-regulatory approaches? What existing authorities would be best positioned to deploy such frameworks, if any?," Federal agencies with expertise in the specific industry sector and related A.I. deployment are already engaging in non-regulatory approaches. NIST, for example, is working with stakeholders to develop a voluntary A.I. Risk Management Framework and other principles and guidance for A.I. to promote trustworthiness.
HHS provides another example , where the Office of the National Coordinator for Health I.T. (ONC) recently convened an A.I. showcase highlighting the health-related A.I. activities of various HHS agencies.
Finally, ongoing dialogue from FSA regulators on this topic is critical to understand the landscape and see how firms are managing risks under current reg frameworks instead of new regulations.
Transparency and Oversight:
In response to question two, "What pre-deployment and post-market assessments should be considered? How can we make such requirements flexible enough to account for various use cases and applications, " NIST's recently finalized guidance, "Towards a Standard for Identifying and Mitigating Bias within AI" (SP 1270), recognizes that bias is not new or unique to AI, recommends various steps to identify and mitigate bias throughout the algorithmic lifecycle, and encourages continuous testing and evaluation of algorithms. Any effort by the Commission should align with ongoing A.I. work both within the federal and with the private sector.
In response to question five, "Is bias in A.I. systems truly eliminable? " we believe it is not possible to have data sets, algorithms, or human decision-making free of all potential biases. The focus should be on identifying and mitigating adverse bias. Also, it is important to highlight that AI in fields such as healthcare can involve research that is predicated on looking for differences. This bias can contribute to better, more precise outcomes for specific populations, particularly historically underserved communities, over algorithms that are overly generalized for a broad population.
In response to question six, "Should the government require companies to be transparent about the use of A.I., and should the government be the clearinghouse for such private information? " we believe that a government clearinghouse for A.I. information creates privacy and security concerns for sensitive data and algorithms. Furthermore, simply reporting A.I. uses may not resolve issues and could lead to an unnecessary burden for companies and organizations. Instead, the focus should be on principles and practices that promote accountability and the responsible development, deployment, and assessment of A.I.
In response to question seven, "If the government is the right clearinghouse, how can the government conduct effective auditing across diverse use cases without unnecessarily revealing proprietary information about a training data set or exactly how a model works? " we again emphasize the support for developing principles and guidelines that are stakeholder-driven that align with other government and agency initiatives in place of auditing use cases post-deployment. This will help mitigate potential issues in the earlier stages of A.I. instead of auditing the vast breadth of A.I. after deployment.
Explainability and Human-in-the-Loop as safety mechanisms:
In response to question one, "What should explainability metrics be considered for federal regulation? Should explainability requirements vary based on application or certain types of risk?" C_TEC would like to highlight NIST's work in developing their "Four Principles of Explainable Artificial Intelligence,[1]" which include "Explanation, Meaningful, Explanational Accuracy and Knowledge limits.[2]" Furthermore, we have long been an advocate of "adopting a risk-based approached to AI-Governance[3]" and believe that risk should play a factor when determining what, if any, metrics should be considered.
In response to question five, "Does requiring a human-in-the-loop somehow shield producers from liability from faulty systems? " more research is necessary. However, we believe it is important to point out that all humans themselves have their own confirmation bias, and interjecting a human into the loop, could, in fact, potentially add bias into the system. Furthermore, while human-in-the-loop may be a useful tool, we believe that other safety mechanisms should also be pursued, requiring meaningful and continual training to guard against bias.
Transparency Requirements and Oversight of Government Use of A.I.:
Regarding question one, "What information should government agencies be required to disclose about their AI systems, and for what purpose? What exemptions, if any, should be made for sensitive use cases? " government agencies should also align with frameworks and principles, such as NIST's pending AI RMF and final principles for identifying and mitigating bias in AI. Sector-specific considerations for AI should also align between the public and private sectors.
Regarding question Two, "What responsibility should the government—versus the private entity that developed the AI—have for the impact of the application on society? What standards should be met before a government agency is allowed to take on a privately-developed system for use? What kinds of impact assessments should be required." werecommend alignment of responsibilities for both public and private AI. This will foster the trust of users and those impacted if expectations are clear and consistent.
Fostering Innovation:
Regarding question three, "How can the U.S. help foster the development and retention of local expertise with AI R&D?" C_TEC strongly supports establishing and strengthening regional hubs throughout the United States to advance workforce, training, representation, and overall digital equity. Webelieveregional innovation centers help develop and meet the needs of those regions. They also can help foster an environment that allows engagement of academic data science R&D talent with industry peers to drive sustained innovation and solution developments in those specific regions.
Regarding question six, "How can the U.S. support R&D, data-sharing, testing, experimentation, regulatory sandboxes, and investment in high-risk A.I. areas?" the U.S. mustcontinue to support both public and private funding and the development of AI R&D opportunities. For example, Anthem's Digital Data Sandbox provides one of the largest certified de-identified health data sets in the U.S. The Anthem Digital Data Sandbox uses certified de-identified data to protect the privacy of individuals. Pursuant to and in accordance with Anthem's issued certificate of de-identification via expert determination, no identifiable Anthem member data is used in the sandbox. This sandbox enables developers to work with applicable U.S. healthcare system information to test their algorithms, which may assist in appropriate testing, updates, and eventual scaling of algorithms to the U.S. healthcare market.
In response to question nine, "How can the U.S. reduce uncertainty around legislative implementation once rules are established? Should the government work together with industry bodies to translate regulatory goals into practical steps for compliance? What bodies are best positioned for this? " we support working with stakeholders to translate regulatory goals into practical steps for implementation and compliance. In the health sector, incorporating input from agencies that leverage health technology considerations into their regulations is ideal. This would include HHS (CMS, ONC, FDA), among others. Standards development organizations such as HL7, a non-profit standards development organization that leverages stakeholder consensus to create data interoperability standards, would also help the larger community to consider appropriate opportunities for data integration.
Furthermore, many different sectors are already highly regulated within this space, such as the financial services sector. For this reason, we believe that it is vitally essential for legislators, regulators, and the business community to work together, as current requirements may suffice, and further regulation may not be necessary.
In response to question ten, "How can the U.S. accelerate the development and adoption of AI quality management systems that legislation may require?" westrongly believe that if any AI quality management system is adopted, it should align with existing work, such as NIST's AI RMF, which will assist in the acceleration of development as well as reduce unnecessary duplication of compliance for organizations.
Conclusion:
C_TEC once again appreciates the opportunity to provide feedback to the Commission on Competitiveness, Innovation, and Inclusion's second request for information. Furthermore, we believe that developing bipartisan and durable policy solutions is extremely important in helping the United States develop a regulatory environment that allows for A.I. to thrive and does not stifle innovation and advancement within the field. We stand ready to work with the Commission and look forward to providing further feedback on your ongoing efforts.
Sincerely,
Michael Richards
Director, Policy
Chamber Technology Engagement Center
U.S. Chamber of Commerce
[1] https://www.nist.gov/publications/four-principles-explainable-artificial-intelligence
[2] https://nvlpubs.nist.gov/nistpubs/ir/2021/NIST.IR.8312.pdf
[3] https://www.uschamber.com/technology/us-chamber-releases-artificial-intelligence-principles
