The disclosure was made in a report submitted to Australia’s eSafety Commission, which monitors online safety and holds tech firms accountable for harm minimisation.

The report, covering April 2023 to February 2024, also revealed that Google received 86 user reports alleging its AI program, Gemini, was being misused to create child exploitation material.

Google confirmed using “hatch-matching,” a system that detects and removes child abuse imagery by comparing newly uploaded images to known harmful content. 

However, no similar system was deployed to identify and eliminate terrorist-related deepfake material, pointing to AI safety inconsistencies.

Since the rise of OpenAI’s ChatGPT in late 2022, governments worldwide have been on the to ensure adequate regulations that prevent AI misuse. 

The European Union has introduced the AI Act, which seeks to regulate high-risk AI applications, while the United States has proposed initiatives like the Blueprint for an AI Bill of Rights to ensure ethical AI development.

Australia’s eSafety Commissioner, Julie Inman Grant, described Google’s report as an important step in understanding the risks associated with AI-generated content. “This underscores how critical it is for companies developing AI products to build in and test the efficacy of safeguards to prevent this type of material from being generated,” she stated.

Challenges in AI Safeguarding

Even with Google’s endeavours to tackle AI-generated child abuse material, the absence of equivalent protections against deepfake terrorist content accentuates the technical and ethical challenges in AI governance. 

Deepfake technology, which allows the creation of highly realistic but fabricated content, could lead to the spread of misinformation, manipulate public opinion, and facilitate fraudulent activities.

The eSafety Commission has previously imposed fines on tech platforms for failing to comply with reporting requirements. Social media platforms X (formerly Twitter) and Telegram have been penalised for not adequately addressing harmful content. 

X was fined A$610,500 ($382,000) and lost its initial appeal but plans to challenge the ruling again, while Telegram is also disputing its penalty.

Tech leaders are willing to spend to reap what they hope will be game-changing business improvements and streamlined operations, with a 61% rise in planned spending on AI in 2024, according to new research.

But business leaders should strike a balance between their excitement for AI with the needs of the business. For all the promise of this technology, many companies have already ended up with AI proof-of-concepts which have not delivered the results they hoped for.

Attaining tangible results from AI investment requires both careful thought, and attention to detail in execution.

There is a huge gap between exciting tech built in the laboratory and the day-to-day reality of business applications. It’s all too easy to take a short-sighted view and become over-excited by technology that has not yet crossed this gap. This is exactly how AI investment is wasted. 

The Risks and Challenges

AI holds significant promise for positive impact in Africa, yet it also brings substantial risks. Unfortunately, these risks may disproportionately affect those least prepared to address them, particularly within the African continent.

Research indicates that AI can perpetuate biases, amplify injustices, and infringe on human rights. The technology has negative implications for transnational organized crime, with criminals exploiting AI to breach personal datasets, applications, and impersonate individuals for extortion or fraud.

In South Africa, impersonation fraud surged by 356% between April 2022 and April 2023.

Moreover, AI can manipulate public opinion through the spread of misinformation or disinformation, a form of soft power.

For instance, during Nigeria’s 2023 general elections, a widely circulated deepfake audio on WhatsApp, Facebook, and other social media platforms falsely portrayed a secret conversation among leaders of the People’s Democratic Party (PDP) planning to compromise the elections.

The advent of advanced deepfake technologies and expansive language models introduces complex socio-technical challenges to democratic processes.

As African countries strive to develop robust AI policies, proactive measures to address AI-related concerns are still in their infancy.

Despite this, many nations on the continent are likely to prioritize staying abreast of AI advancements and harnessing its potential benefits. This focus may obscure potential threats and postpone solutions to future challenges.

Strategic Approaches for AI Governance in Africa

African countries can design and adopt their own governance approaches to address the current and future roles of AI through several key strategies:

Develop comprehensive national AI strategies: While African countries can begin to implement AI strategies to capitalize on their potential immediately, they should prioritize the creation of comprehensive national AI strategies.

These strategies should encompass regulations, investment opportunities, and consideration of country-specific peculiarities while aligning with existing continental and global policies.

Integrate AI Policies with broader initiatives: AI-related policy activities and interventions should be woven into broader initiatives focused on privacy, security, data access, intellectual property protection, human rights, and cross-border data interchange.

A coordinated effort by diverse stakeholders is necessary to connect these policy areas and develop a coherent action plan for policymakers.

Foster partnerships and knowledge sharing: African governments should prioritize partnerships and knowledge-sharing opportunities to advance AI initiatives on the continent.

This involves collaborating with other governments, regional organizations, academic institutions, and industry players to exchange best practices, resources, and expertise in AI governance.

Identify and mitigate AI risks: Governments and partners should begin to identify, amplify, and mitigate potential AI risks. This can be achieved by prioritizing ethical AI education and increasing awareness of AI-related harms.

Integrating AI ethics and responsible AI principles into educational curricula and raising public awareness about the implications of AI technologies are essential steps.

Given the evolving nature of AI, continuous research is crucial to track emerging threats and provide early warning support.

AI Adoption and Innovation

African consumers, educational institutions, governments, and companies are rapidly adopting AI to aid in content creation, improve the delivery of public services, and streamline business processes.

Although there is limited information on consumer adoption of AI tools across the continent, usage has surged due to the introduction of ChatGPT, DALL-E, Midjourney, and other commercial AI tools.

According to the 2024 Stanford AI Index, 27% of Kenyans use ChatGPT daily, placing Kenya third behind India and Pakistan.

Google search trends also reveal rising consumer interest in AI across Africa, with searches related to AI increasing by 270% over the last year and 400% over the last five years in countries such as Kenya.

African educational institutions, such as the University of Pretoria in South Africa, Makerere University in Uganda, and Kwame Nkrumah University of Science and Technology in Ghana, have developed prominent AI labs and published research focused on leveraging AI for social impact.

The African Institute for Mathematical Sciences (AIMS), launched in 2003 with educational centers in Cameroon, Ghana, Senegal, South Africa, and Rwanda, has established several Master’s and PhD programs in AI, machine learning, math, and data science, graduating over 3,000 students.

Additionally, large tech companies have established AI research labs in Africa. Microsoft set up the Microsoft Africa Research Institute in Nairobi, Kenya, in 2020. Google launched an AI research lab in Accra, Ghana, in 2018, and IBM Research opened labs in Nairobi, Kenya, in 2013 and Johannesburg, South Africa, in 2016. These initiatives reflect a significant commitment to advancing AI research and application across the continent

Staying Agile in the Ever-Evolving Era of AI

Business leaders must stay strategic when dealing with AI and avoid getting caught up in the excitement surrounding the technology.

This requires taking a step back to focus on real, tangible problems and prioritizing which ones to address first.

A holistic approach is essential: AI integration should be seamlessly woven into the solution for a real-life business challenge, and the project should be something that multiple teams can actively engage with.

By adopting a measured, holistic approach, leaders can ensure that AI projects move beyond the planning stage and truly harness the benefits of this transformative technology.

Deepfake (“deep learning” + “fake”) artificial intelligence has been growing at a rapid rate over the past few years. Machine learning can be used to create compelling fakes of images, video, or audio content.

To determine whether some audio piece is a fake or a speech of a real human, there are several characteristics to consider: the timbre, manner and intonation of speech.

For example, a voice deepfake will give out an unnatural monotony of speech. Another feature that should be considered is the sound quality.

So, illegible speech and strange noises should be alerted while listening to an audio message or a call.

Deepfake technology in itself is harmless, but in the hands of scammers it can become a dangerous tool with lots of opportunities for deception, defamation or disinformation. Fortunately, there haven’t been any mass cases of scams involving voice alteration, but there have been several high-profile cases involving voice deepfakes.

In 2019, scammers used this technology to shake down a UK-based energy firm. In a telephone conversation, the scammer pretended to be the chief executive of the firm’s German parent company and requested an urgent transfer of €220,000 to the account of a certain supplier company. A year later, in 2020, scammers used deepfakes to steal up to $35,000,000 from a Japanese company.

“Currently the technology for creating high-quality deepfakes is not available for widespread use. However, in the future, it may become freely open, which could lead to a surge in related fraud. Most likely, attackers will try to generate voices in real time – to impersonate someone’s relative and lure out money, for example. Such a scenario is not realistic for now: creating high-quality deepfakes involves a lot of limited resources. However, to make a low-quality audio fake, fewer resources are required, and fraudsters can use this. These signs can be helpful to spot this fraud,” comments Dmitry Anikin, Senior Data Scientist at Kaspersky.

To protect yourself from deepfakes, Kaspersky experts recommend:

• Pay attention to suspicious calls. You should be alerted by poor sound quality, unnatural monotony of voice, unintelligible speech, extraneous noise.
• Don’t make decisions based on emotions, don’t share your details with anyone, and don’t transfer money, even if the interlocutor sounds convincing. It is better to stop the call and double-check the information received through several channels.
• Use reliable security solutions on your devices, which will further secure the use of gadgets.

The prices for creating or purchasing deepfakes may vary depending on the complexity of the project and the quality of the final product.

According to the estimates by Kaspersky experts, prices per one minute of a deepfake video may range from $300 to $20,000.

This was announced at the Kaspersky Cyber Security Weekend – META 2023.

Kaspersky analysed various Darknet marketplaces and underground forums offering creation of deepfake videos and audios for different malicious purposes. In some cases, individuals may request specific targets for deepfake creation, such as celebrities or political figures.

Cybercriminals use generative AI video in several ways for illegal activities. They can use deepfakes to create fake videos or images that can be used to defraud individuals or organisations. For example, they can create a fake video of a CEO requesting a wire transfer or authorising a payment, which can be used to steal corporate funds.

Fake videos can be used to create compromising videos or images of individuals, which can be used to extort money or information from them.

Cybercriminals can also use deepfakes to spread false information or manipulate public opinion. For example, they can create a fake video of a politician making controversial statements, which can be used to influence the outcome of an election.

Deepfake technology can be employed to bypass verification in payment services by creating realistic fake videos or audio recordings of a legitimate account owner. These can be used to trick payment service providers into thinking that they are the actual account owner, thereby gaining access to the account and its associated funds.

“Increasingly, deepfakes are used in attempts at blackmail and fraud. For example, the CEO of a British energy firm was tricked out of $243,000 by a voice deepfake of the head of his parent company requesting an emergency transfer of funds. As a result, funds were wired to the fraudster’s bank account. Suspicions were only raised when the criminal requested another transfer, but by then it was too late to get the funds that were already transferred back. A similar case was reported in the UAE, where $400,000 were stolen in a scam also involving voice deepfake,” comments Vladislav Tushkanov, Lead Data Scientist at Kaspersky.

“It’s important to remember that deepfakes are a threat not only to businesses, but also to individual users: they can spread misinformation, be used for scams, or to impersonate someone without consent. Increasing your digital literacy level is key to counter these threats.”

– Tushkanov

Continuous monitoring of dark web resources provides valuable insights into the deepfake industry, allowing researchers to track the latest trends and activities of threat actors in this space. By monitoring the darknet, researchers can uncover new tools, services, and marketplaces used for the creation and distribution of deepfakes.

This type of monitoring is a critical component of deepfake research and helps to improve our understanding of the evolving threat landscape. Kaspersky’s Digital Footprint Intelligence service includes this type of monitoring to help its customers stay ahead of the curve when it comes to deepfake-related threats.

The technology giant is not the only one to regulate deepfakes — several US states have laws regulating them, China draft law requires identification of computer-generated media, and the prospective EU AI regulation may include a clause about this particular technology.

Kaspersky experts have explained what deepfakes are, and why there is so much controversy around them – along with how users can protect themselves:

• ‘Deepfakes’ commonly refer to various kinds of synthetic, computer-generated media involving people and made with deep neural networks. These might be videos, photos or voice recordings. The use of deep learning, instead of the traditional image editing techniques, drastically lowers the effort and skill required to create a convincing fake.
• Originally, the term referred to a particular piece of software which had gained popularity on Reddit. The software could implant the face of a person into a video featuring another person, and was used almost entirely to create non-consensual porn with celebrities. According to some estimates, up to 96% of all deepfakes are pornographic, highlighting concerns around deepfakes being used for abuse, extortion and public shaming.
• This technology could also aid cybercriminals. In at least two cases, in England and United Arab Emirates, a voice deepfake has been used to trick companies into transferring funds to fraudsters, by posing as the officers of the respective companies. Recent research showed that commercial liveness detection algorithms, used by financial institutions in KYC procedures, could be tricked by deepfakes created from ID photos, creating new attack vectors and making leaked IDs an even more serious issue.
• Another issue is that deepfakes undermine trust in audio and video content as they can be used for nefarious purposes. For example, in a recent case a fake interview of Elon Musk was used to promote a cryptocurrency scam. Various experts and institutions, such as Europol, warn that the growing availability of deepfakes can lead to further proliferation of disinformation on the Internet.
• It’s not all bad news. Image manipulation is as old as the images themselves, and CGI has been around for decades, and both have found decent use, as have deepfakes. For example, in a recent Kendrick Lamar’s video, Heart Part 5, deepfake technology was used to morph the rapper’s face into other famous celebrities, such as Kanye West. In the movie Top Gun: Maverick, an algorithm was used to voice Val Kilmer’s character after the actor lost his voice. A deepfake algorithm was also used to create a viral TikTok series starring fake Tom Cruise. And a few startups are looking for new ways to use the tech, for example, to generate lifelike metaverse avatars.

With all the trust issues surrounding deepfakes, users can wonder how to spot a deepfake. Here are some tips to get started:

• A convincing deepfake, such as the one featuring Tom Cruise, still requires a lot of expertise and effort — and sometimes even a professional impersonator. Deepfakes used for scams still tend to be low-quality and can be spotted by noticing unnatural lip movements, poorly rendered hair, mismatched face shapes, little to no blinking, skin color mismatches and so on. Errors in the rendering of clothes or a hand passing over the face can also give away an amateur deepfake.
• If you see a famous or public person making wild claims or too-good-to-be-true offers, even if the video is convincing, proceed with due diligence and cross-check the information with reputable sources. Note that fraudsters can intentionally code down videos to hide the deficiencies of deepfakes, so the best strategy is not staring at the video looking for clues, but using your common sense and fact-checking skills.
• A trusted security solution can provide good support if a high-quality deepfake convinces users to download any malicious files or programs, or to visit any suspicious links or phishing websites.
• In case you are the victim of deepfake porn, you can reach out to both the website to ask for the video to be taken down (many websites prohibit posting deepfakes), and to a law enforcement agency, as generating deepfakes is a criminal offence in some legislations.

“Deepfakes are a prime example of a technology that develops faster than we can actually comprehend and manage the complications. This is why it is perceived both as an addition to an artist’s toolkit as well as a new disinformation instrument that challenges what we as a society can trust”, says Vladislav Tushkanov, lead data scientist at Kaspersky.