WEBVTT
Kind: captions
Language: en-GB

1
00:00:00.120 --> 00:00:05.339
European law protects you from automated decisions.
Which sounds like a minor thing, but if an

2
00:00:05.339 --> 00:00:09.389
automated system is making a meaningful decision
about your life -- whether you get a pay rise,

3
00:00:09.389 --> 00:00:13.450
or if you get a formal warning for clocking
in late at work -- you must be told that it

4
00:00:13.450 --> 00:00:18.210
was an automated decision, and you must be
able to get it reviewed by a human. At least,

5
00:00:18.210 --> 00:00:22.850
in Europe. And importantly, you can tell a
company to never make automated decisions

6
00:00:22.850 --> 00:00:27.940
about you and they must comply with that.
Now that doesn't apply to little things like

7
00:00:27.940 --> 00:00:33.190
online personality quizzes, unless that quiz
decides -- without a human in the loop -- whether

8
00:00:33.190 --> 00:00:35.710
you're a safe person to lend money to.

9
00:00:35.710 --> 00:00:39.579
But those are all fairly benign automated
systems, even if it might not feel like it

10
00:00:39.579 --> 00:00:41.649
when you've just been turned down for a loan.

11
00:00:41.649 --> 00:00:49.170
What about automated systems that are meant
to be harmful to humans?

12
00:00:49.170 --> 00:00:53.940
I've filmed at this tank once before. It's
an old Soviet T-34 battle tank, put here in

13
00:00:53.940 --> 00:00:59.760
London two decades ago by an angry local resident
who was denied permission to build on this

14
00:00:59.760 --> 00:01:04.190
bit of land... but did have permission to
store vehicles here. Or maybe he got permission

15
00:01:04.190 --> 00:01:08.110
to put a "tank" there, because the local council
thought he was talking about some sort of

16
00:01:08.110 --> 00:01:13.170
storage tank, not an actual T-34. The story's
got a bit muddled over the last twenty years or so.

17
00:01:13.580 --> 00:01:19.080
But this tank is designed for people. It's
designed for human soldiers making human decisions.

18
00:01:19.080 --> 00:01:22.870
And that concept might not be the case for
very much longer.

19
00:01:22.870 --> 00:01:29.490
See, the US Army just put out a report called
Visualising the Tactical Ground Battlefield

20
00:01:29.490 --> 00:01:34.460
in the Year 2050. Which sounds like a really
dull comic book, but it's actually a pretty

21
00:01:34.460 --> 00:01:40.660
good read, and they have some things to say
about automated decision making.

22
00:01:40.660 --> 00:01:45.020
The US Army believes that humans will be incapable
of handling all the information that'll be

23
00:01:45.020 --> 00:01:49.400
thrown at them in the height of a future battle.
Computer processing power, they say, won't

24
00:01:49.400 --> 00:01:53.730
be a limiting factor, and as humans grow more
accustomed to automated decision making, of

25
00:01:53.730 --> 00:01:58.520
course we'll be happy to use that in the battlefield
too. If you have a soldier who's happy with

26
00:01:58.520 --> 00:02:03.500
flipping the autopilot on their car, of course
they're going to be happy doing exactly the

27
00:02:03.500 --> 00:02:06.600
same for their tank and for their gun.

28
00:02:06.600 --> 00:02:10.069
"Battle rhythm," -- and I'm quoting the report
directly here,

29
00:02:10.069 --> 00:02:14.409
"Battle rhythm will increase to the point
that, in many instances, humans will no longer

30
00:02:14.409 --> 00:02:19.329
be able to be 'in the loop,' but will instead
need to operate 'on the loop', where 'humans

31
00:02:19.329 --> 00:02:24.959
can only observe the behaviors that are taking
place'... and 'act after the fact or in anticipation

32
00:02:24.959 --> 00:02:26.879
of expected behaviors.'"

33
00:02:26.879 --> 00:02:32.799
So just to be clear: the US Army believes,
in the next few decades, that they will have

34
00:02:32.799 --> 00:02:38.019
autonomous weapons which will have patterns
of behaviour that an engineer is going to

35
00:02:38.019 --> 00:02:43.680
have to try and debug while it is firing live
ammunition. If you're a programmer, working

36
00:02:43.680 --> 00:02:48.219
on open-source computer vision, or collision
avoidance, or, say, something like flocking

37
00:02:48.219 --> 00:02:53.810
behaviour algorithms, there is a chance that
your code will end up as part of that. And

38
00:02:53.810 --> 00:02:58.519
if you've got moral objections: well, good
for you, but it's probably not going to stop anyone.

39
00:02:58.909 --> 00:03:05.700
And I suspect all the legislation that Europe
can muster won't do a thing when you've got

40
00:03:05.700 --> 00:03:08.680
one of these, quite literally, calling the
shots.

41
00:03:09.680 --> 00:03:11.680
[Translating these subtitles? Add your name here!]