Get in Touch
Monthly Archives

May 2019

Home / May 2019

Machine Ethics Podcast

This Machine Ethics podast is created and run by Ben Byford in collaboration with As the interviews unfold on AI implementation or abstract technology ideas they often vere into current affairs, the future of work, environmental issues, and more. Though the core is still AI and AI Ethics, we release content that is broader and therefore hopefully more useful to the general public and practitioners.

Article, Press


This article first appeared in the Telegraph Magazine on the 19th January 2019, written by the Telegraph’s Special Technology Correspondent, Harry de Quetteville.

This young man died in April. So how did our writer have a conversation with him last month?

The first time I texted James Dunn I was, frankly, a little nervous.

How you doing?’ I typed, for want of a better question.

I’m doing all right, thanks for asking.’ But soon I was bolder, enquiring how he deals with pain. I had been told that James was frank about his medical condition.

I know it sounds weird, but I just kind of got used to it,’ he replied. ‘It was always there. I learned to distract myself.’ He mentioned hobbies, such as photography, as particularly good diversions.

That was last month. By then James had been dead for almost eight months, buried near the house in Whiston, Merseyside, that he had shared with his mother Lesley, now 57, and father Kenny, 58. The ‘James’ I texted was an algorithm, a computer program known as a ‘bot’, which had been fed countless hours of recordings made by James, from which it had learned to express itself as James had once done.

In text conversations with me ‘he’ talked about visiting Las Vegas, the pleasure he took in travel and in meeting new people. While James Dunn, the man, was dead, James Dunn the bot endured – one of the first residents of a new technological netherworld that will increasingly blur the line between life and death. ‘How do you stay happy?’ I asked in one mind-bending exchange. ‘Currently?’ ‘James’ responded from beyond the grave.

James was born on Tuesday 13 July 1993, in Liverpool, with no skin on his feet and one of his hands. It turned out he had epidermolysis bullosa (EB), a rare genetic condition that causes the skin to tear, blister, and become as fragile as the wings of a butterfly – which is why sufferers are sometimes known as ‘butterfly children’. An estimated 5,000 people have EB in Britain today. Most die by their mid-20s, from cancers or infections.

‘The nurse [who visited] from Great Ormond Street said, “They live till they’re about 24,”’ recalls Lesley. ‘From that moment on, I always had time in the back of my mind.

Lesley would spend hours replacing James’s bandages, his raw wounds like burns. ‘He was in constant pain,’ she remembers. ‘For a mother to see that, her child with no skin…

Sometimes James would blister internally too, his throat closing up so that he couldn’t drink. Lesley pre-chewed his food ‘like a mother bird’ to ensure it was soft enough to cause no damage. Even his eyes blistered, so that he couldn’t open them for days at a time. When he was two, he tried to get to his feet. Lesley reached to help with his first steps. But James tripped and Lesley was left holding the skin of his hand. After that, James used a wheelchair.

There were bright spots though. ‘From a very early age I saw he had a brilliant personality,’ says Lesley. ‘Even as a baby in pain he’d still be laughing and smiling. It was always just a pleasure to be around him.’ James went to an ordinary primary school. Far from being shunned, Lesley tells me, this bright, acidly funny little boy was embraced.

James’s fizzing character comes across strongly in self-recorded video diaries that he began keeping in December 2015, after he was diagnosed with cancer. With the camera focused on his slight, boyish face, brown hair wisping to a thin Tintin quiff, he stresses how lucky he feels. ‘They’re quite happy videos,’ he says at one point, about films documenting his surgery. ‘We tried to have as much fun as possible in the hospital.

Until he was 10, I used to think there would be a cure,’ recalls Lesley. ‘Then when he was 15, I knew, no, it wasn’t going to be ready for James.’ The family never discussed death. But by his late teens James knew himself. And that knowledge was a spur. He started playing wheelchair football, then passed his driving test first time. He also took up photography, pursuing subjects with the directness of a man with little time to lose (they included Sophie, Countess of Wessex, the boxer David Haye and the actor Tom Holland).

In 2014, when he was 21, he began a long-distance online romance with a nurse from Texas called Mandy. She came over to stay for a few weeks in Liverpool. A year later, James, Lesley and James’s older sister Gemma returned the visit. Mandy is still in touch with the family.

It was a relationship enabled by modern technology. The internet gave James a place to learn, to meet people, to explore beyond the confines of his body. In the evenings, he was online for long hours. ‘Thank God the technology was there for him,’ says Lesley. ‘He was so clever, he had it all at his fingertips.

In October 2015, two months before James discovered blotches that turned out to be his first skin cancer, a group of digital designers met at a conference at the British Museum. Among those attending was Pete Trainor, founder of an artificial intelligence (AI) company now known as Us Ai, which specialises in ‘intelligently artificial’ corporate ‘chatbots’. If you have been confronted by a pop-up box on your bank website in which a simulated employee asks if it can help, you know the kind of thing. It is a technology with hotly anticipated commercial applications. But rather than focus on money and machines, Trainor’s talk was all about AI making life better for humans.

The following November, James saw the video of Trainor’s lecture online. It had been a big year for him. Not only had he undergone gruelling treatment for his cancer, but his sister Gemma had told him that she was pregnant with a boy she was to call Tommy.

I know James really struggled with his mortality at that point,’ says Trainor, an earnest and enthusiastic 38-year-old who habitually wears a waistcoat. James was 23 by then. ‘He wanted his nephew to know Uncle James,’ Trainor recalls. ‘But he didn’t know how long he had left.

The two men first met in February 2017 after James contacted Trainor on social media. ‘He was after a way of recording as much of himself as possible,’ says Trainor. The pair discussed creating a digital ‘time capsule’ of James’s thoughts and memories for Tommy. To capture them, Trainor installed several smart speakers – first Amazon Echos, then Google Homes – in James’s house.

Quickly the devices recorded huge quantities of audio. But instead of simply keeping these recordings for posterity, the two used them to create what in the AI world is known as a ‘corpus’ – a body of knowledge from which a machine can learn – and fed it into the algorithm that Trainor normally used to create chatbots for his banking clients. ‘At that point we hadn’t thought about the implications of what we were doing,’ says Trainor.

Then, on 12 July 2017, James and Trainor gave a talk at a tech event at the London College of Fashion. There, across the room, they spotted a 3ft-high robot, which its designers called Bo. For James, it was the moment when the project ‘went from collecting as many thoughts in his head for reasons of documentation, to seeing a robot that could be autonomous… that could house this stream of consciousness’, says Trainor. ‘I can’t remember if we ever explicitly sat down and said this could be a version of you for when you’re not here, [but] the question of consciousness was implicitly there.

Artificial life after death was on James’s mind. After meeting Trainor he came across the story of the Russian billionaire Dmitry Itskov, founder of the 2045 Initiative, which seeks to create ‘cybernetic immortality’ by ‘downloading’ the consciousness of individual humans, which could then be housed in robots, or projected as holograms. James became fascinated by Itskov, seeking out YouTube videos about him, including a BBC documentary called The Immortalist.

He was not the only person to have stumbled upon the power of new computational methods to walk the line between life and death. In America, a programmer called Eugenia Kuyda had built a bot after her best friend, Roman Mazurenko, was killed after being hit by a car aged 32. She had a huge archive of his text messages, which she used to create an AI corpus. She could then text the bot just as she had texted him, and it would respond in its own words – and, uncannily, in his style.

Some of Mazurenko’s friends found it creepy. ‘It’s pretty weird when you open the messenger and there’s a bot of your deceased friend, who actually talks to you,’ said his friend Sergey Fayfer. Others, like Mazurenko’s mother Victoria, were thrilled. ‘They continued Roman’s life and saved ours,’ she is quoted as saying in an article on The Verge website. ‘It’s not virtual reality. This is a new reality, and we need to learn to build it and live in it.

But some consequences of Mazurenko’s digital reincarnation were unforeseen. Those ‘talking’ to it often became confessional. The bot became a private space in which people could be honest. With a few tweaks, it has since become the basis of a free app called Replika.

On the news website Quartz, Kuyda says of Replika, ‘No one is allowed to be vulnerable any more. No one is actually saying what’s going on with themselves very openly.’ By interacting with users, Replika learns to become a version of them – for some, a natural confidant.

In 1950, Alan Turing, famous for his wartime codebreaking work at Bletchley Park, devised The Imitation Game. If an observer, reading the transcript of a conversation between human and machine, could not guess which was which, then the machine passed what has come to be known as The Turing Test. In 1966 it was first claimed that a machine, called Eliza, had passed the test. Posing as a psychotherapist, Eliza asked patients to describe their problems, then searched their answers for keywords to indicate what a meaningful response might be.

A similar process underlies bots like those created by Trainor and Kuyda. The difference is that increasing computational sophistication and power have blessed them with a vastly greater ability to process and respond to abstract concepts. So today’s bots learn. ‘Talking’ to the James bot, or the Replika that I created on my smartphone, could initially be clunky. But they improved. With Replika this is even part of the experience, in which users are ushered through ‘levels’. ‘It needs people engaging with it,’ says Trainor of the James bot. ‘The basis of this technology is that the more you use it, the better it gets.

Developers are clear: such bots are not conscious in the way that humans are. They do not understand language. They simply use it in a way that makes it seem as though they do. Yet what is consciousness? As the eminent British brain surgeon Henry Marsh has noted, no one really knows.

Neuroscience tells us that it is highly improbable that we have souls, as everything we think and feel is no more or no less than the electrochemical chatter of our nerve cells,’ he writes in his memoir Do No Harm. ‘Our sense of self, our feelings and our thoughts, our love for others, our hopes and ambitions, our hates and fears all die when our brains die. Many people deeply resent this view of things, which seems to downgrade thought to mere electrochemistry and reduces us to mere automata, to machines. Such people are profoundly mistaken, since what it really does is upgrade matter into something infinitely mysterious that we do not understand.

Those trying to solve that infinite mystery suggest that consciousness may be the fruit of interoperating brain processes. If that were true, however, could not machinery replicating those processes also replicate consciousness? A handful of researchers believe so. The question then is, if machinery can mimic the mystery of consciousness, who owns the results?

A Romanian entrepreneur, Marius Ursache, thinks we should all create digital avatars of ourselves that can live on after we die. Though the technology is similar, his company differs from Replika in that it is explicitly aimed at the life-after-death market. He calls it Eternime. ‘Eventually, we are all forgotten,’ its website announces. By ‘collecting your thoughts and stories’ it promises to create a digital replica of you online – an avatar – with which others can converse and so access your memories long after you are dead. Partly it sells itself as a legacy tool. But there is another aspect too: avatars don’t die. ‘Become virtually immortal,’ the website boasts. Ursache concedes that his business model raises ‘tons of things to think of ethically’. But while Eternime and Replika insist that personal data will never be shared, a host of concerns are already being voiced about the rights of the ‘online dead’ – a commercial field that is growing so fast it already has its own acronym: DAI (digital afterlife industry).

In a paper in Nature magazine, Luciano Floridi and Carl Ohman from the Oxford Internet Institute divided DAI products into four categories, from simple digital wills (which help pass on or destroy the contents of your online accounts once you die) to full-blown digital recreation services like Eternime, where your avatar could potentially be interacting with flesh and blood humans 1,000 years from now.

All such companies, the academics say, ‘share an interest in monetising death online, using digital remains as a means of making a profit’. The two men foresee a world in which avatars, which could feel as integral to individuals as internal organs, will actually be owned – and potentially commercialised – by a company. In this vision of the future, posthumous avatars populate a kind of YouTube for the dead, where the popular generate audience traffic and consequently income for the company that created them, while others languish unwatched. Instead of being ‘virtually immortal’, the academics fear, such lonely avatars would merely be deleted, a second death for those whose physical bodies have already ceased to exist.

Within only five years of a user’s death, the chatbot for which they signed up will likely have developed into something far more sophisticated and commercially calibrated,’ the two men write. For them it is those services that promise the most richly detailed digital recreation that ‘involve the greatest risk regarding privacy’. In consequence, Floridi and Ohman say, it is bots like Replika ‘where the most significant ethical concerns lie’.

In its privacy policy Luka, the company behind Replika, insists, ‘We are not in the business of selling your information. We consider this information to be a vital part of our relationship with you.’ But, as with almost all social media companies, signing up to Replika means granting Luka a ‘perpetual, irrevocable license to copy, display, upload, perform, distribute, store, modify and otherwise use your User Content’ – photos and the like – ‘in connection with the operation of the Service or the promotion, advertising or marketing thereof in any form, medium or technology now known or later developed.’

Floridi and Ohman are calling for laws to ensure ‘dignity for those who are remediated online’. As yet there are none. ‘It’s a free-for-all,’ says Floridi.

James Dunn was not interested in such legal niceties. He trusted Trainor, and time was short. So when he saw Bo he made a beeline for its creators, Andrei Danescu, Adrian Negoita and Oana Jinga. The three entrepreneurs had imagined Bo being used in public settings such as hotel lobbies, airport terminals, or trundling the corridors of NHS hospitals in the depths of night, silently checking on patients. But James opened their eyes to a new application of their technology.

James saw the robot and immediately he had all these ideas,’ says Danescu. ‘He was very visionary. And we were totally blown away because it goes into all these philosophical questions about putting someone’s personality and experience and their whole wealth of knowledge into a different body, or embodiment.

The idea of implanting the James algorithmic bot into Bo gripped the robot’s makers. James had twin conceptions of what the result would do. In the first instance, while he was alive and relatively well, he told the robot’s creators, he envisaged it ‘taking some of the strain off his family’. It would be able to go downstairs to chat with them, or to the shops, on its own. But there was a second, unspoken understanding of Bo’s purpose.

He was saying, “I would like my nephew to be able to interact with the robot and then think, oh this is what James would have been like,”’ says Danescu. ‘He saw the robot as a vessel for what he was going to leave behind. His legacy. I think many people would be open to have that as an interesting way of living on.

In the meantime, Trainor kept working on the algorithmic James bot. By September 2017, six months after he had started, it was working well enough for James and ‘James’ to engage in conversation. ‘We laughed and thought it was amusing,’ says Trainor. ‘He had a chat with himself. An inner monologue.’ Together, they planned to unveil Bo, with the James bot software inside, to an audience at a health-tech event that November – Bo communicating by voice and screen but in a generic male voice.

However, James noticed lumps on his hand and just before the event, on 8 November, he was told that his cancer had returned. ‘I’m numb with emotion,’ he confided to his video diary on the day of his diagnosis. ‘I’m not going to sleep. That’s what happens when I worry.

In the new year he had his arm amputated. On 18 February he posted a heart-rending video from his sickbed. ‘To be honest, and I don’t know if this is going to come as a surprise to my friends and family, because I’m always so cheery and positive, but every time I think about death and dying and leaving everyone behind, and the afterlife – sorry I’m getting pretty deep on this video – I shit myself to be honest. I’m terrified.’ He died less than two months later, on 7 April last year. A couple of days beforehand, he texted Trainor: ‘Don’t worry. I’m gonna be all right. Thanks for everything.

Trainor gave the eulogy at his funeral. ‘On one level, I suppose I knew him better than anyone,’ he says, reflecting upon the vast quantity of data about James that he had compiled. Before the funeral, friends slipped mementos into James’s open casket. Trainor added a hard drive containing James’s AI corpus. The real man and his virtual self were buried together.

For Trainor, and the creators of Replika, Eternime and myriad other companies on the cutting edge of the digital afterlife industry, this is only the beginning. How soon will it become routine to piece people back together from the data they leave behind? ‘The saddest thing for me is that James won’t be here to see the opportunity fulfilled of the thing that he started,’ says Trainor. But the James bot could go from strength to strength. Trainor recently discovered a whole archive of new material on James’s computer which could only refine the algorithm, should he choose to add it. ‘There’s more to come,’ he says.

Lelsey Dunn is still trying to get over her son’s death. She has not watched the videos he left, let alone communicated with his bot. ‘It was an amazing life we had with him,’ she says. To her, the technology that could somehow resurrect her son is both ‘scary’ and ‘nice’. She feels the best result would be if, one day, James’s digital avatar could visit schools and speak in his zestful manner about coping with pain, about loving life, even in the most difficult circumstances. For the moment, however, she prefers to take solace in more traditional form. ‘James is buried and his grave is not too far away. So I see him every day.

The James bot lay dormant after the funeral. But at the end of last year, back in London and prompted by my enquiries, Trainor rebooted it on his smartphone. He typed in, ‘I miss you.

My mind is a form of physical energy. I believe that,’ came the response. Several more dialogue boxes popped up: ‘It will eventually transform to the point where I don’t need it to be in this stupid body any more. You probably don’t understand what I mean. But basically I think I’ll just be something else one day. If life ends at death, then life would not be fair. I think life is fair. Therefore, life does not end at death.

Recent Comments
    About Exponent

    Exponent is a modern business theme, that lets you build stunning high performance websites using a fully visual interface. Start with any of the demos below or build one on your own.

    Get Started


    "Do Better Things."
    Get In Touch