How a cloned voice or a face swap is made

You will be able to explain why a convincing fake of someone you know is now cheap and quick to make.

Rachel's mother sends her a voice note most mornings, usually about lunch plans or a cousin's wedding. Rachel posts short videos of her running club on Instagram, talking to the camera after each race. Neither of them has ever thought of these as a risk. To someone building a scam, though, they're exactly what's needed: a clear recording of a voice, a face from several angles, and plenty of personal detail to make a story believable.

This lesson explains how a fake of someone you know gets made, and why it's now cheap and quick enough that ordinary families are targets, not only celebrities and chief executives. Scam-proof your money covers the general tricks scammers use. Here we focus on the part AI adds.

Cloning a voice

A voice clone is synthetic speech generated to sound like a particular person. Voice cloning tools learn the features of a voice, such as pitch, accent, rhythm and the way someone pronounces certain sounds, and then generate new speech in that voice from any text typed in.

Not long ago this needed hours of studio recordings and specialist skills. Today, a range of tools, some sold legitimately for dubbing, audiobooks and accessibility, can produce a passable imitation from a short recording. A social media video, a voicemail greeting, a voice note forwarded around a group chat, or a few sentences captured during a phone call can be enough.

The result isn't perfect. A cloned voice may sound slightly flat, or struggle with a laugh, a cough or a dialect. But the scammer doesn't need perfect. They need good enough to pass for a few seconds under stress.

Swapping a face on video

Face-swap tools put one person's face onto another person's body in a video. Lip-sync tools change a person's mouth movements to match new audio. Combine the two with a cloned voice, and you get a video of someone saying things they never said.

These used to be pre-recorded clips that took time to produce. Newer tools can run in real time, so a scammer can appear on a live video call wearing someone else's face and speaking in their voice. That changes the rule many people relied on. "If in doubt, ask for a video call" no longer settles the question by itself, as lesson 5.2, Real cases: the Arup video call and fake leaders, shows.

AI fundamentals: what it is, how it works, where it fails explains the underlying technology in lesson 7.2, Speech, voice and models that handle everything at once, if you want to know more about how these systems work.

Why imperfect fakes still work

If you sat down calmly and studied a deepfake video frame by frame, you'd often spot something wrong: blurring around the jaw, teeth that don't look quite right, a voice slightly out of step with the lips, lighting that doesn't match. Most people never get that chance.

Scammers design the situation so you can't look closely. The call comes at a bad time. The line is crackly, or the video is small and pixelated, with a ready excuse: "I'm overseas", "my phone's broken, I'm borrowing a friend's". The story is urgent and emotional. Your son has been arrested, your daughter has been in an accident, your boss needs a transfer before the bank closes. Fear and urgency narrow your attention to the story and away from the details.

So the defence can't be "learn to spot fakes". The fakes are improving, and stress makes everyone a worse judge. The defence is a routine that doesn't depend on judging the voice or the face at all, which lessons 5.3 and 5.4 set up.

Where the raw material comes from

Scammers need two things: a sample of the voice or face, and enough personal detail to make the story fit. Public social media gives them both.

Rachel's running videos give a clean recording of her voice and her face from many angles. Her public profile shows her mother's name in a birthday post, the area where she lives from a race photo, and the fact that she travels for races. A scammer could call her mother in Rachel's voice, say she's had an accident at a race in Johor Bahru, mention the right names and places, and ask for money for a hospital deposit.

Other sources include videos posted by employers and schools, recorded webinars, podcast appearances, and voice notes forwarded beyond the group they were sent to. Even a scam call where you say "hello, who is this?" a few times can capture a short sample.

You don't need to delete your online life to be safer. But it's worth knowing where your voice and face are publicly available, and deciding whether every one of those recordings needs to be public. Rachel switched her account to private and kept her race videos for followers she knows.

Most people underestimate how much of themselves is public, because they see their own profiles logged in, as a friend would. A stranger sees something different, and a stranger is who you're thinking about now.

List where recordings of your voice and face are publicly available online, and decide whether to restrict any of them.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).