12898.
For purposes of this chapter, the following definitions apply:(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b) “Automated decision system” means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. “Automated decision
system” does not include a spam email filter, firewall, antivirus software, identity and access management tools, calculator, database, dataset, or other compilation of data.
(c) “Legally protected information” means information that a person is prohibited from disclosing under federal or state law, including provisions of the Evidence Code relating to privilege, or that would result in a violation of a legal duty of confidentiality.
(d) “State agency” has the same meaning as in Section 11000.
(e) “Personally identifiable information” means an individual’s residential address, telephone number, social security number, driver’s license number, state identification card number, passport number, license plate number, vehicle registration information for a motor vehicle owned or leased by the individual, and information
regarding an individual’s precise geolocation, as defined in Section 1798.140 of the Civil Code.
(f) “Protected health information” has the same meaning as defined in Section 160.103 of Title 45 of the Code of Federal Regulations.
(g) “Services” means both of the following:
(1) Services, benefits, or assistance, whether provided in cash or in kind, that a state agency provides or administers, including, but not limited to, social services, linkages to programs administered by the federal Social Security Administration, vocational and education-related services, and employment assistance.
(2) Issuance, renewal, denial, or suspension of a professional license or occupational credential.
12898.1.
If a state agency uses an automated decision system for services, not including competitive determinations, the state agency shall comply with all of the following:(a) The state agency may use an automated decision system to inform its decisionmaking process. The state agency shall not substitute the outputs of an automated decision system for human judgment.
(b) When an automated decision system is used to assist in a decisionmaking process, the system shall be only one of the factors a user considers in reaching a decision. The state agency may use an automated decision system to ascertain whether a services application or submission meets minimum eligibility thresholds as predetermined by the
state agency.
(c) The state agency shall not use an output from an automated decision system as the sole basis for an adverse service determination affecting a natural person, such as denial of a benefit or license, except as expressly authorized by federal or state law.
(d) The state agency shall require that any output of an automated decision system that suggests noneligibility or other adverse action be reviewed by a human before any adverse action is taken.
(e) A state agency’s user shall not represent work generated solely by an automated decision system as the user’s own original work.
(f) When the use of an automated decision system is material to a decision, the state agency shall provide a means, consistent with applicable law, to document or
disclose that the system was used in the decisionmaking process.
(g) The state agency shall verify the accuracy of an automated decision system’s outputs, and shall promote nondiscrimination in its use of an automated decision system, by doing all of the following:
(1) Ensure content, recommendations, or other outputs generated by an automated decision system that may materially affect service levels are reviewed and verified by an employee of the state agency, or by another authorized person, for accuracy before being relied upon.
(2) Monitor and periodically evaluate the use of automated decision systems to reduce the risk that outputs contain or perpetuate bias, including bias based on race, color, religion, sex, gender, gender identity, gender expression, sexual orientation, marital status, national origin,
ancestry, age, disability, medical condition, genetic information, immigration or citizenship status, or any other characteristic protected by federal or state law.
(3) Require that an application or submission contain all required fields, attachments, or information in the required format.
(h) The state agency shall safeguard personally identifiable information, protected health information, or other legally protected information by prohibiting a user of the automated decisions system from inputting, uploading, or otherwise disclosing the information to an automated decision system, except where necessary for services administration or delivery, as authorized by law and subject to appropriate safeguards. For third-party systems, a state agency shall employ safeguards that may include access controls and appropriate security standards.
(i) The state agency’s director or designee shall provide for an initial and subsequent periodic quality control review of the outputs of the automated decision system, or a statistically valid represented sample thereof to assure acceptable accuracy.