WEBVTT
Kind: captions
Language: en

1
00:00:00.380 --> 00:00:03.380
This is a brilliant tweet.

2
00:00:03.380 --> 00:00:05.160
But I don't want you to pay attention
to the tweet.

3
00:00:05.160 --> 00:00:07.990
It's good, sure, but I want you to watch the
numbers that are underneath it.

4
00:00:07.990 --> 00:00:09.700
That's a screen recording,

5
00:00:09.700 --> 00:00:13.090
and the numbers are going up and down,
all over the place.

6
00:00:13.090 --> 00:00:15.400
They should steadily rise, but they don’t.

7
00:00:15.400 --> 00:00:20.440
There aren't that many people tapping ‘like’
by mistake and then taking it back.

8
00:00:20.440 --> 00:00:23.480
So why can't Twitter just count?

9
00:00:26.029 --> 00:00:28.159
You'll see examples like this
all over the place.

10
00:00:28.159 --> 00:00:32.610
On YouTube, subscriber and view counts
sometimes rise and drop seemingly at random,

11
00:00:32.610 --> 00:00:36.620
or they change depending on which device you're
checking on.

12
00:00:36.620 --> 00:00:39.230
Computers should be good at counting, right?

13
00:00:39.230 --> 00:00:41.820
They're basically just overgrown calculators.

14
00:00:41.820 --> 00:00:43.150
This video that you're watching,

15
00:00:43.150 --> 00:00:46.800
whether it's on a tiny little phone screen
or on a massive desktop display,

16
00:00:46.800 --> 00:00:50.290
it is all just the result of
huge amounts of math that turns

17
00:00:50.290 --> 00:00:54.300
a compressed stream of binary numbers into
amounts of electricity

18
00:00:54.300 --> 00:00:58.510
that get sent to either a grid of coloured pixels
or a speaker,

19
00:00:58.510 --> 00:01:00.310
all in perfect time.

20
00:01:00.310 --> 00:01:02.910
Just counting should be easy.

21
00:01:02.910 --> 00:01:05.399
But sometimes it seems to fall apart.

22
00:01:05.399 --> 00:01:07.469
And that's usually when there's a
big, complicated system

23
00:01:07.469 --> 00:01:09.520
with lots of inputs and outputs,

24
00:01:09.520 --> 00:01:12.399
when something has to be done at scale.

25
00:01:12.399 --> 00:01:15.020
Scaling makes things difficult.
And to explain why,

26
00:01:15.020 --> 00:01:20.790
we have to talk about race conditions, caching,
and eventual consistency.

27
00:01:20.790 --> 00:01:24.329
All the code that I've talked about in The Basics
so far has been single-threaded,

28
00:01:24.329 --> 00:01:27.689
because, well, we’re talking about the basics.

29
00:01:27.689 --> 00:01:29.829
Single-threaded means that it looks like a
set of instructions

30
00:01:29.829 --> 00:01:32.149
that the computer steps through
one after the other.

31
00:01:32.149 --> 00:01:36.560
It starts at the top, it works its way through,
ignoring everything else,

32
00:01:36.560 --> 00:01:39.789
and at the end it
has Done A Thing.

33
00:01:39.789 --> 00:01:41.610
Which is fine, as long as
that's the only thread,

34
00:01:41.610 --> 00:01:43.549
the only thing that the computer's doing,

35
00:01:43.549 --> 00:01:45.800
and that it's the only computer doing it.

36
00:01:45.800 --> 00:01:47.279
Fine for old machines like this,

37
00:01:47.279 --> 00:01:51.469
but for complicated, modern systems,
that’s never going to be the case.

38
00:01:51.469 --> 00:01:55.249
Most web sites are, at their heart, just a
fancy front end to a database.

39
00:01:55.249 --> 00:01:57.130
YouTube is  a database of videos and comments.

40
00:01:57.130 --> 00:01:58.829
Twitter is a database of small messages.

41
00:01:58.829 --> 00:02:04.100
Your phone company's billing site is a
database of customers and bank accounts.

42
00:02:04.100 --> 00:02:08.440
But the trouble is that a single computer
holding a single database can only deal with

43
00:02:08.440 --> 00:02:10.650
so much input at once.

44
00:02:10.650 --> 00:02:15.580
Receiving a request, understanding it, making
the change, and sending the response back:

45
00:02:15.580 --> 00:02:17.350
all of those take time,

46
00:02:17.350 --> 00:02:21.280
so there are only so many requests
that can fit in each second.

47
00:02:21.280 --> 00:02:24.260
And if you try and handle
multiple requests at once,

48
00:02:24.260 --> 00:02:27.670
there are subtle problems that can show up.

49
00:02:27.670 --> 00:02:30.330
Let's say that YouTube wants to count
one view of a video.

50
00:02:30.330 --> 00:02:33.540
It just has the job of adding one
to the view count.

51
00:02:33.540 --> 00:02:37.300
Which seems really simple, but it's actually
three separate smaller jobs.

52
00:02:37.300 --> 00:02:39.090
You have to read the view count,

53
00:02:39.090 --> 00:02:40.450
you have to add one to it,

54
00:02:40.450 --> 00:02:43.250
and then you have to write that view count
back into the database.

55
00:02:43.250 --> 00:02:45.930
If two requests come along
very close to each other,

56
00:02:45.930 --> 00:02:47.750
and they’re assigned to separate threads,

57
00:02:47.750 --> 00:02:52.000
it is entirely possible that the second thread

58
00:02:52.000 --> 00:02:53.670
could read the view count

59
00:02:53.670 --> 00:02:57.510
while the first thread is still doing its calculation.

60
00:02:57.510 --> 00:03:02.010
And yeah, that's a really simple calculation,
it's just adding one,

61
00:03:02.010 --> 00:03:04.980
but it still takes a few ticks of a processor.

62
00:03:04.980 --> 00:03:09.790
So both of those write processes would put
the same number back into the database,

63
00:03:09.790 --> 00:03:11.980
and we've missed a view.

64
00:03:11.980 --> 00:03:15.050
On popular videos, there'll be collisions
like that all the time.

65
00:03:15.050 --> 00:03:18.620
Worst case, you've got ten or a hundred of those
requests all coming in at once,

66
00:03:18.620 --> 00:03:21.110
and one gets stuck for a while
for some reason.

67
00:03:21.110 --> 00:03:24.220
It'll still add just one to the original number
that it read,

68
00:03:24.220 --> 00:03:25.500
and then, much later,

69
00:03:25.500 --> 00:03:27.980
it'll finally write its result back into the
database.

70
00:03:27.980 --> 00:03:30.020
And we've lost any number of views.

71
00:03:30.020 --> 00:03:34.620
In early databases, having updates that collided
like that could corrupt the entire system,

72
00:03:34.620 --> 00:03:37.510
but these days things will generally at least
keep working,

73
00:03:37.510 --> 00:03:40.210
even if they're not quite accurate.

74
00:03:40.210 --> 00:03:42.540
And given that YouTube has to work out not
just views,

75
00:03:42.540 --> 00:03:44.380
but ad revenue and money,

76
00:03:44.380 --> 00:03:46.720
it has got to be accurate.

77
00:03:46.720 --> 00:03:49.550
Anyway, that’s a basic race condition:

78
00:03:49.550 --> 00:03:52.240
when the code’s trying to do two or more
things at once,

79
00:03:52.240 --> 00:03:56.060
and the result changes depending on the order
they occur in,

80
00:03:56.060 --> 00:03:59.380
an order that you cannot control.

81
00:03:59.380 --> 00:04:01.780
One solution is to put all the requests in
a queue,

82
00:04:01.780 --> 00:04:06.030
and refuse to answer any requests until the
previous one is completed.

83
00:04:06.030 --> 00:04:08.960
That's how that single-threaded, single-computer
programming works.

84
00:04:08.960 --> 00:04:10.620
It's how these old machines work.

85
00:04:10.620 --> 00:04:14.569
Until the code finishes its task and says
"okay, I'm ready for more now",

86
00:04:14.569 --> 00:04:17.110
it just doesn't accept anything else.

87
00:04:17.110 --> 00:04:19.390
Fine for simple stuff, does not scale up.

88
00:04:19.390 --> 00:04:24.990
A million-strong queue to watch a YouTube video
doesn't sound like a great user experience.

89
00:04:24.990 --> 00:04:28.830
But that still happens somewhere, for
things like buying tickets to a show,

90
00:04:28.830 --> 00:04:34.110
where it'd be an extremely bad idea to accidentally
sell the same seat to two people.

91
00:04:34.110 --> 00:04:37.780
Those databases have to be 100% consistent,
so for big shows,

92
00:04:37.780 --> 00:04:39.830
ticket sites will sometimes start a queue,

93
00:04:39.830 --> 00:04:43.090
and limit the number of people accessing the
booking site at once.

94
00:04:43.090 --> 00:04:49.900
If you absolutely must count everything accurately,
in real time, that’s the best approach.

95
00:04:49.900 --> 00:04:52.950
But for sites dealing with Big Data, like
YouTube and Twitter,

96
00:04:52.950 --> 00:04:56.100
there is a different solution called
eventual consistency.

97
00:04:56.100 --> 00:04:58.800
They have lots of servers all over the world,

98
00:04:58.800 --> 00:05:03.080
and rather than reporting every view
or every retweet right away,

99
00:05:03.080 --> 00:05:05.600
each individual server will keep its own count,

100
00:05:05.600 --> 00:05:08.919
bundle up all the viewcounts and statistics
that it’s dealing with,

101
00:05:08.920 --> 00:05:12.920
and just it will just update the central system
when there's time to do so.

102
00:05:12.920 --> 00:05:14.460
Updates doesn't have to be hours apart,

103
00:05:14.460 --> 00:05:16.440
they can just be minutes
or even just seconds,

104
00:05:16.450 --> 00:05:21.360
but having a few bundled updates that can
be queued and dealt with individually

105
00:05:21.360 --> 00:05:23.169
is a lot easier on the central system

106
00:05:23.169 --> 00:05:26.940
than having millions of requests all being
shouted at once.

107
00:05:26.940 --> 00:05:28.930
Actually, for something on YouTube’s scale,

108
00:05:28.930 --> 00:05:31.490
that central database won't just be one computer:

109
00:05:31.490 --> 00:05:34.180
it'll be several, and they'll all be keeping
each other up to date,

110
00:05:34.180 --> 00:05:39.160
but that is a mess we really don't want
to get into right now.

111
00:05:39.160 --> 00:05:42.020
Eventual consistency isn't right for everything.

112
00:05:42.020 --> 00:05:46.730
On YouTube, if you're updating something like
the privacy settings of a video,

113
00:05:46.730 --> 00:05:50.850
it's important that it's updated immediately
everywhere.

114
00:05:50.850 --> 00:05:54.160
But compared to views, likes and comments,
that's a really rare thing to happen,

115
00:05:54.160 --> 00:05:57.690
so it's OK to stop everything,
put everything else on hold,

116
00:05:57.690 --> 00:06:02.000
spend some time sorting out that
important change, and come back later.

117
00:06:02.000 --> 00:06:04.400
But views and comments,
they can wait for a little while.

118
00:06:04.400 --> 00:06:07.700
Just tell the servers around the world to
write them down somewhere, keep a log,

119
00:06:07.700 --> 00:06:12.130
then every few seconds, or minutes, or maybe
even hours for some places,

120
00:06:12.130 --> 00:06:13.949
those systems can run through their logs,

121
00:06:13.949 --> 00:06:19.270
do the calculations and update the
central system once everyone has time.

122
00:06:19.270 --> 00:06:23.800
All that explains why viewcounts and
subscriber counts lag sometimes on YouTube,

123
00:06:23.800 --> 00:06:26.310
why it can take a while to get
the numbers sorted out in the end,

124
00:06:26.310 --> 00:06:32.340
but it doesn't explain the up-and-down numbers
you saw at the start in that tweet.

125
00:06:32.340 --> 00:06:34.160
That's down to another thing: caching.

126
00:06:34.160 --> 00:06:38.690
It's not just writing into the database that's
bundled up. Reading is too.

127
00:06:38.690 --> 00:06:41.340
If you have thousands of people requesting
the same thing,

128
00:06:41.340 --> 00:06:44.220
it really doesn't make sense to have them all
hit the central system

129
00:06:44.220 --> 00:06:46.720
and have it do the calculations
every single time.

130
00:06:46.720 --> 00:06:52.199
So if Twitter are getting 10,000 requests
a second for information on that one tweet,

131
00:06:52.200 --> 00:06:54.800
which is actually a pretty reasonable
amount for them,

132
00:06:54.800 --> 00:06:59.540
it'd be ridiculous for the central database to
look up all the details and do the numbers every time.

133
00:06:59.540 --> 00:07:02.320
So the requests are actually going to a cache,

134
00:07:02.320 --> 00:07:04.420
one of thousands, or maybe
tens of thousands of caches

135
00:07:04.420 --> 00:07:07.630
sitting between the end users
and the central system.

136
00:07:07.630 --> 00:07:10.940
Each cache looks up the details
in the central system once,

137
00:07:10.940 --> 00:07:13.400
and then it keeps the details in its memory.

138
00:07:13.400 --> 00:07:16.139
For Twitter, each cache might only keep them
for a few seconds,

139
00:07:16.139 --> 00:07:18.210
so it feels live but isn't actually.

140
00:07:18.210 --> 00:07:22.400
But it means only a tiny fraction
of that huge amount of traffic

141
00:07:22.400 --> 00:07:24.490
actually has to bother the central database:

142
00:07:24.490 --> 00:07:27.860
the rest comes straight out of memory
on a system that is built

143
00:07:27.860 --> 00:07:29.540
just for serving those requests,

144
00:07:29.540 --> 00:07:31.140
which is orders of magnitude faster.

145
00:07:31.150 --> 00:07:33.270
And if there's a sudden spike in traffic,

146
00:07:33.270 --> 00:07:35.360
Twitter can just spin up some more cache servers,

147
00:07:35.360 --> 00:07:37.600
put them into the pool that's
answering everyone's requests,

148
00:07:37.600 --> 00:07:42.380
and it all just keeps working without
any worry for the database.

149
00:07:42.380 --> 00:07:46.340
But each of those caches will pull that information
at a slightly different time,

150
00:07:46.340 --> 00:07:47.990
all out of sync with each other.

151
00:07:47.990 --> 00:07:51.300
When your request comes in, it's routed to
any of those available caches,

152
00:07:51.300 --> 00:07:54.580
and crucially it is not going to be
the same one every time.

153
00:07:54.580 --> 00:07:56.690
They've all got slightly different answers,

154
00:07:56.690 --> 00:08:00.580
and each time you're asking a different one.

155
00:08:00.580 --> 00:08:05.280
Eventual consistency means that everything
will be sorted out at some point.

156
00:08:05.280 --> 00:08:10.040
We won't lose any data, but it might take
a while before it's all in place.

157
00:08:10.040 --> 00:08:14.550
Sooner or later the flood of retweets will
stop, or your viewcount will settle down,

158
00:08:14.550 --> 00:08:17.900
and once the dust has settled
everything can finally get counted up.

159
00:08:17.900 --> 00:08:22.600
But until then:
give YouTube and Twitter a little leeway.

160
00:08:22.600 --> 00:08:26.340
Counting things accurately is really difficult.

161
00:08:26.340 --> 00:08:29.840
Thank you very much to the Centre for
Computing History here in Cambridge,

162
00:08:29.840 --> 00:08:31.720
who've let me film with all this
wonderful old equipment,

163
00:08:31.720 --> 00:08:34.560
and to all my proofreading team
who made sure my script's right.