• As Artificial Intelligence (AI) expands into new parts of our economy, new regulations in areas such as the healthcare sector are being proposed in 2024. 
• In the healthcare sector, there are many concerns about the privacy and accuracy of AI algorithms, both from governments and independent organizations. 
• As policy makers look to the future of AI, they need to consider the balance between regulation and innovation in the healthcare sector. 

Introduction 

As Artificial Intelligence (AI) expands into different sectors of United States society, deciding how to regulate AI while still allowing innovation is a pressing problem. In the healthcare sector, there have been multiple lawsuits regarding the use of AI. These span both the insurance and care provider industries, with concerns of bias in AI systems, faulty medical recommendations, and negative influence on practitioners’ decisions. As these problems arise, many states have begun to propose and implement regulations in this industry. Additionally, the federal government has released statements regarding these problems, and other influential bodies have released recommendations for regulations. Policy makers, especially federal policy makers, need to consider state and federal AI regulations and determine which course of action will both protect patients and encourage innovation. 

Federal Regulations 

In October 2023, President Biden signed an executive order relating to AI in many sectors. Included in this were promises to expand AI research in the healthcare sector, and to advance responsible use of AI in healthcare. In December 2023, the White House followed up on these promises, having 28 leading healthcare providers and payers signing a commitment to use AI that is Fair, Appropriate, Valid, Effective, and Safe (FAVES). Other parts of these commitments include continuing to “vigorously” develop AI in the healthcare sector, inform users if AI content has not been checked by a human, and create a framework for monitoring AI usage and mitigating potential harm.  

Also in December 2023, the Department of Health and Human Services (HSS) released a final ruling of the  Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing release. This rule establishes a standard for the healthcare industry surrounding AI, including transparency regarding AI usage and adopting United States Core Data for Interoperability Version 3 as a baseline for the certification of AI. Transparency standards include clinicians’ ability to access baseline information about the algorithm they are using, and to consistently test the algorithm to ensure “fairness, appropriateness, validity, effectiveness, and safety.”   

State Regulations 

In the 2024 legislative session, twelve states  have proposed bills regarding AI regulation in the healthcare sector, with others expected to follow. Illinois, Massachusetts, and Rhode Island proposed bills that require the informed consent of patients going to mental health professionals for the use of AI in their treatment. Both Rhode Island and Massachusetts also require the mental health professional to obtain permission to use AI from the proper licensing board. New York, Oklahoma, and Pennsylvania proposed bills requiring health insurance companies to disclose their use of AI on their website, to healthcare providers, and to covered persons. Insurance companies must also take steps to minimize bias within the algorithm and datasets. California proposed a bill that requires a licensed physician to supervise algorithms used for requests or denials by providers for authorization. Illinois proposed a bill that requires algorithms and datasets to be submitted to the Department of Public Health and the Department of Innovation and Technology, requires that the AI is proven to be as good or better as other diagnostic options, and requires that patients be given other diagnostic options that do not involve AI and get informed consent from the patient to use AI in their diagnosis. Florida proposed a bill that will establish a Creates the Health Care Innovation Council, a panel of experts that will promote safe innovation of AI in the healthcare sector and set up a loan system for companies that require money to implement new AI technology. Hawaii proposed a bill that will add AI to the “emerging issues” considered by the State Health Planning and Development Agency. 

NCOIL and NAIC Actions and Concerns 

In 2022, the National Association of Insurance Commissioners (NAIC) published an article regarding policy and regulatory considerations of AI in the insurance space. The main risks highlighted are the risk of bias in AI algorithms, especially bias based on the data they are trained on. This poses the risk of people’s insurance coverage being based on trends in their racial or gender profiles, rather than based on their need. Additionally, the NAIC brings up the problem of insurance companies using unregulated third-party algorithm developers, which increases the opacity between the training the model goes through and the provider using the model. They also address the problem of patient data protection, which may not be guaranteed when using a third-party algorithm.  

In December 2023, the NAIC approved a model bulletin  regarding the use of AI by health insurers. The model emphasizes the need for AI regulation in the insurance industry for any use of AI, including underwriting, product development, and fraud detection. It also encourages risk management for the use of AI, including testing and retesting AI outputs, ensuring suitable data is used to train the AI, monitoring protection of consumer information, and creating an approval process for the use of AI. 

The National Conference of Insurance Legislators (NCOIL) published a comment on the NAIC’s model, highlighting both their agreement and concerns. The NCOIL agrees that regulation is needed for AI in the insurance space due to the problems the NAIC mentions. However, they propose that the term “bias” should not be used in the model. “Bias” is not a term used in insurance regulation, as insurances companies are able to discriminate and adjust rates based on certain characteristics of the person receiving the insurance. In a way, bias is a core part of the insurance industry. They note that the term “unfair bias” does not have a strict definition that can mitigate this problem. They also argue that the term “algorithm” is too broadly defined. The NAIC definition could apply to multiple algorithms that have been used in the insurance space for years, and do not pose the same problems as new AI algorithms and their usage. Without a narrow definition, insurance companies may end up being required to rework or get rid of current algorithms that are not analogous in type or objective to the new AI algorithms this model is aimed at regulating. 

Future Federal Policy 

As states seem to be moving towards regulating AI, federal policy makers need to consider if there is a need for them to make overarching legislation. Most states that have enacted or are planning on enacting AI regulations in the healthcare space are doing so through bills that apply to one specific space, such as insurance or mental healthcare. This can potentially leave holes in the spaces that are not regulated. Additionally, the white house is pushing for greater innovation, and focusing less on regulation. The federal government needs to consider how some of these state regulations may affect innovation of AI in the healthcare spaces. Although these preliminary bills do not fully restrict AI use, the possibility of harsher bills in the future, and potential all-out bans of AI in the healthcare space could completely stifle innovation and prevent patients from getting a higher level of care. This may eventually cause certain states with lesser regulations to make greater advancements in the healthcare space than states with greater regulations. These AI algorithms also help with the burden of necessary but tedious paperwork that both insurers and care providers spend time creating and filling out. States with less AI regulation in this space may experience an increase in productivity and happiness rates among people working in these industries, as they can focus more on patient interaction and less on paperwork. This will make some states more attractive to work in than others, potentially changing the quality of healthcare in certain areas.  

Federal policy makers need to decide what the best case of action is to assure that patients are getting high quality care that is keeping up with new technology, while ensuring the safety and fairness of the use of AI in this sector. 

Conclusion 

As many states are proposing and passing AI regulation legislation, it is important to consider the long-term effects of the extent of AI usage in the healthcare space. Many states are concerned with algorithms being trained with biased data, or with decision making algorithms diagnosing patients without a doctor checking the results. They are also concerned with users’ privacy and the right to know when an algorithm is being used, both in the insurance sector and the care provider sector. Additionally, President Biden released an executive order committing to the advancement of AI in the healthcare space (among other spaces) and had 28 healthcare companies agree to develop AI as much as possible while still committing to safe and fair treatment of patients. Other independent organizations, such as the NCOIL and the NAIC have expressed concerns with AI usage in this sector and have proposed regulation models for state legislators to use specifically in the insurance industry. Their concerns echo that of the federal government, specifically noting potential privacy problems and unfair treatment of patients. Federal policy makers need to consider the balance between innovation and regulation in this space. Heavy regulation may slow down advancements in AI’s usage in healthcare that could potentially save lives and lower the workload for doctors and insurance providers. Lack of regulation may allow patients to be exploited and receive incorrect treatment. As the country moves to adapt with this new age of technology, a balance must be struck between these extremes.