Personal pictures of Brazilian children have been misused to power artificial intelligence tools, global watchdog Human Rights Watch reported Monday. Pictures of children are being included without their knowledge or consent in a data set that companies are using for AI tools to be trained. These AI tools are subsequently used to create deep fakes using the children’s likeness derived from the images, which HRW alleges puts other children at risk of being exploited and harmed.

An analysis conducted by the NGO found that a popular data set for the training of AI tools called LAION-5B contains links to photos of Brazilian children. Some of the captions and URLs that accompany the photos of the children have information, such as the name and location of the hospital where they were born, as well as their location at the time the picture in question was taken. HRW was able to find 170 photos of children from 10 different states in Brazil and went on to say that this is most likely the tip of the iceberg; only 0.0001 percent of the images and captions contained in the dataset were reviewed.

HRW Brazil stated on X that it is “very clear” that neither the children in the images nor their parents were aware of their child’s likeness being used for AI. In an interview with Brazilian newspaper Folha de S.Paulo, HRW children’s rights and technology researcher Hye Jung Han  discussed the report’s findings, saying:

Children should not have to live in fear that their photos might be stolen and weaponized against them. The government should urgently adopt policies to protect children’s data from AI-fueled misuse.

The pictures contained in the dataset appeared to be private, with many of them originally posted to personal blogs not readily accessible through a simple online search. The AI tools learn to create life-like, realistic-looking images based on the likeness of any child in these images, which in turn can be abused by people to create explicit imagery of the children concerned.

Brazilian Lawmakers have proposed to ban the use of AI to generate sexually explicit images of people, including children without consent. Han stated, “Protecting children’s data privacy now will help to shape the development of this technology into one that promotes, rather than violates, children’s rights.” She added that Brazil’s Congress should ensure that all proposed AI regulations should incorporate data privacy protection, especially for children.

With the allegations made against LAION-5B, the nonprofit that manages the dataset, LAION, stated that they would remove the images of the children. LAION stated that the best way to protect against abuse and misappropriation is by ensuring that children and their guardians do not post any personal photos to the internet.