On November 13, 2024, FinCEN issued an alert to help financial institutions identify fraud schemes associated with the use of deepfake media created with generative artificial intelligence (GenAI) tools. The alert explains typologies associated with these schemes, provides red flag indicators to assist with identifying and reporting related suspicious activity, and reminds financial institutions of their reporting requirements under the Bank Secrecy Act.
Since 2023, FinCEN has noted a rise in suspicious activity reports from financial institutions about deepfake media being used in fraud schemes. These schemes often involve criminals creating or altering identity documents to bypass verification and authentication. GenAI tools have significantly lowered the resources needed to create high-quality synthetic content such as "deepfakes," which can be highly realistic and difficult to distinguish from human-generated material.
FinCEN’s analysis of BSA data indicates that criminals have used GenAI to create falsified documents, photographs, and videos to bypass financial institutions’ customer identification and verification and customer due diligence controls. When investigating a suspected deepfake, reverse image searches, open-source research, and advanced techniques like metadata analysis or deepfake detection software can help identify potential manipulations. Indicators that further scrutiny may still be needed include:
Inconsistencies among multiple identity documents submitted by the customer;
A customer’s inability to satisfactorily authenticate their identity, source of income, or another aspect of their profile; and
Inconsistencies between the identity document and other aspects of the customer’s profile.
Financial institutions have also detected deepfake identity documents through enhanced due diligence on accounts showing other suspicious activity, with certain indicators prompting further investigation such as:
Access to an account from an IP address that is inconsistent with the customer’s profile;
Patterns of apparent coordinated activity among multiple similar accounts;
High payment volumes to potentially higher-risk payees, such as gambling websites or digital asset exchanges;
High volumes of chargebacks or rejected payments;
Patterns of rapid transactions by a newly opened account or an account with little prior transaction history; and
Patterns of withdrawing funds immediately after deposit and in manners that make payments difficult to reverse in cases of suspected fraud, such as through international bank transfers or payments to offshore digital asset exchanges and gambling sites.
FinCEN identified the following red flag indicators to help financial institutions detect, prevent, and report potential suspicious activity related to the use of GenAI tools for illicit purposes:
A customer’s photo is internally inconsistent (e.g., shows visual tells of being altered) or is inconsistent with their other identifying information (e.g., a customer’s date of birth indicates that they are much older or younger than the photo would suggest).
A customer presents multiple identity documents that are inconsistent with each other.
A customer uses a third-party webcam plugin during a live verification check. Alternatively, a customer attempts to change communication methods during a live verification check due to excessive or suspicious technological glitches during remote verification of their identity.
A customer declines to use multifactor authentication to verify their identity.
A reverse-image lookup or open-source search of an identity photo matches an image in an online gallery of GenAI-produced faces.
A customer’s photo or video is flagged by commercial or open-source deepfake detection software.
GenAI-detection software flags the potential use of GenAI text in a customer’s profile or responses to prompts.
A customer’s geographic or device data is inconsistent with the customer’s identity documents.
A newly opened account or an account with little prior transaction history has a pattern of rapid transactions; high payment volumes to potentially risky payees, such as gambling websites or digital asset exchanges; or high volumes of chargebacks or rejected payments.
Read FinCEN’s press release here.
The full Alert can be found here.