1 00:00:00,030 --> 00:00:02,400 The following content is provided under a Creative 2 00:00:02,400 --> 00:00:03,840 Commons license. 3 00:00:03,840 --> 00:00:06,840 Your support will help MIT OpenCourseWare continue to 4 00:00:06,840 --> 00:00:10,530 offer high quality educational resources for free. 5 00:00:10,530 --> 00:00:13,390 To make a donation or view additional materials from 6 00:00:13,390 --> 00:00:17,490 hundreds of MIT courses, visit MIT OpenCourseWare at 7 00:00:17,490 --> 00:00:21,470 ocw.mit.edu. 8 00:00:21,470 --> 00:00:22,130 PROFESSOR: OK. 9 00:00:22,130 --> 00:00:24,480 I want to start where we left off. 10 00:00:24,480 --> 00:00:30,200 You remember last time we were looking at Fibonacci. 11 00:00:30,200 --> 00:00:32,700 And so we've got it up here, a nice little recursive 12 00:00:32,700 --> 00:00:34,970 implementation of it. 13 00:00:34,970 --> 00:00:40,110 And the thing I wanted to point out is, we've got this 14 00:00:40,110 --> 00:00:43,140 global variable number of calls. 15 00:00:43,140 --> 00:00:47,210 Which is there not because Fibonacci needs it but just 16 00:00:47,210 --> 00:00:50,600 for pedagogical reasons, so that we can keep track of how 17 00:00:50,600 --> 00:00:53,180 much work this thing is doing. 18 00:00:53,180 --> 00:00:55,590 And I've turned on a print statement which 19 00:00:55,590 --> 00:00:57,330 was off last time. 20 00:00:57,330 --> 00:01:01,120 So we can see what it's doing is it runs. 21 00:01:01,120 --> 00:01:18,200 So let's try it here with Fib of 6. 22 00:01:18,200 --> 00:01:25,210 So, as we would hope, Fib of 6 happens to be 8. 23 00:01:25,210 --> 00:01:28,660 That right? 24 00:01:28,660 --> 00:01:30,290 That right, everybody? 25 00:01:30,290 --> 00:01:34,360 Should Fib of 6 be 8? 26 00:01:34,360 --> 00:01:38,270 I don't think so. 27 00:01:38,270 --> 00:01:42,030 So first thing we should do is scratch our heads and see 28 00:01:42,030 --> 00:01:48,710 what's going on here. 29 00:01:48,710 --> 00:01:52,950 Alright, let's look at it. 30 00:01:52,950 --> 00:01:57,100 What's happening here? 31 00:01:57,100 --> 00:02:04,870 This is your morning wake-up call. 32 00:02:04,870 --> 00:02:13,170 What is happening? 33 00:02:13,170 --> 00:02:13,510 Yes. 34 00:02:13,510 --> 00:02:22,840 STUDENT: [INAUDIBLE] 35 00:02:22,840 --> 00:02:25,070 PROFESSOR: See if I can get it all the way to the back. 36 00:02:25,070 --> 00:02:26,450 No I can't. 37 00:02:26,450 --> 00:02:28,270 That's embarrassing. 38 00:02:28,270 --> 00:02:29,770 Alright. 39 00:02:29,770 --> 00:02:34,330 So if n is less than or equal to 1, return n. 40 00:02:34,330 --> 00:02:39,150 Well that's not right, right? 41 00:02:39,150 --> 00:02:44,360 What should I be doing there? 42 00:02:44,360 --> 00:02:48,530 Because Fib of 0 is? 43 00:02:48,530 --> 00:02:49,680 What? 44 00:02:49,680 --> 00:02:50,630 1. 45 00:02:50,630 --> 00:03:12,250 So let's fix it. 46 00:03:12,250 --> 00:03:14,180 How about that, right? 47 00:03:14,180 --> 00:03:20,220 Or maybe, what would be even simpler than that? 48 00:03:20,220 --> 00:03:23,740 Maybe I should just do that. 49 00:03:23,740 --> 00:03:31,620 Now let's try it. 50 00:03:31,620 --> 00:03:36,760 We feel better about this? 51 00:03:36,760 --> 00:03:39,240 We like that answer? 52 00:03:39,240 --> 00:03:46,330 Yes, no? 53 00:03:46,330 --> 00:03:47,450 What's the answer, guys? 54 00:03:47,450 --> 00:03:51,850 What should Fibonacci of 6 be? 55 00:03:51,850 --> 00:03:55,920 I think that's the right answer, right? 56 00:03:55,920 --> 00:03:58,520 OK. 57 00:03:58,520 --> 00:04:03,060 So what do I want you to notice about this? 58 00:04:03,060 --> 00:04:07,100 I've computed a value which is 13. 59 00:04:07,100 --> 00:04:10,810 And it's taken me 25 calls. 60 00:04:10,810 --> 00:04:14,400 25 recursive calls to get there. 61 00:04:14,400 --> 00:04:17,110 Why is it taking so many? 62 00:04:17,110 --> 00:04:22,640 Well, what we can see here is that I'm computing the same 63 00:04:22,640 --> 00:04:26,030 value over and over again. 64 00:04:26,030 --> 00:04:28,560 Because if we look at the recursive structure of the 65 00:04:28,560 --> 00:04:40,880 program, what we'll see that's going on here is, I call Fib 66 00:04:40,880 --> 00:04:47,280 of 5 and 4, but then Fib of 5 is also going 67 00:04:47,280 --> 00:04:49,170 to call Fib of 4. 68 00:04:49,170 --> 00:04:54,210 So I'm going to be computing it on those branches. 69 00:04:54,210 --> 00:04:59,120 And then it gets worse and worse as I go down. 70 00:04:59,120 --> 00:05:02,090 So if I think about computing Fib of 0 I'm going to be 71 00:05:02,090 --> 00:05:04,880 computing that a lot of times. 72 00:05:04,880 --> 00:05:07,710 Now, fortunately, Fib of 0 is short. 73 00:05:07,710 --> 00:05:10,690 But the other ones are not so short. 74 00:05:10,690 --> 00:05:15,930 And so what I see is that as I run this, I'm doing a lot of 75 00:05:15,930 --> 00:05:18,540 redundant computation. 76 00:05:18,540 --> 00:05:28,140 Computing values whose answer I should already know. 77 00:05:28,140 --> 00:05:32,180 That's, you'll remember last time, I talked about the 78 00:05:32,180 --> 00:05:36,910 notion of overlapping sub-problems. And that's what 79 00:05:36,910 --> 00:05:40,310 we have here. 80 00:05:40,310 --> 00:05:45,210 As with many recursive algorithms, I solve a bigger 81 00:05:45,210 --> 00:05:48,710 problem by solving a smaller instance of 82 00:05:48,710 --> 00:05:50,940 the original problem. 83 00:05:50,940 --> 00:05:52,750 But here there's overlap. 84 00:05:52,750 --> 00:05:56,750 The instant unlike binary search, where each instance 85 00:05:56,750 --> 00:06:01,340 was separate, here the instances overlap. 86 00:06:01,340 --> 00:06:03,910 They share something in common. 87 00:06:03,910 --> 00:06:07,940 In fact, they share quite a lot in common. 88 00:06:07,940 --> 00:06:12,030 That's not unusual. 89 00:06:12,030 --> 00:06:16,110 That will lead me to use a technique I mentioned again 90 00:06:16,110 --> 00:06:24,600 last time, called memoization. 91 00:06:24,600 --> 00:06:35,970 Effectively, what that says is, we record a value the 92 00:06:35,970 --> 00:06:50,520 first time it's computed, then look it up the subsequent 93 00:06:50,520 --> 00:06:56,300 times we need it. 94 00:06:56,300 --> 00:06:57,220 So it makes sense. 95 00:06:57,220 --> 00:06:59,850 If I know I'm going to need something over and over again, 96 00:06:59,850 --> 00:07:03,470 I squirrel it away somewhere and then get it 97 00:07:03,470 --> 00:07:06,800 back when I need it. 98 00:07:06,800 --> 00:07:25,750 So let's look at an example of that. 99 00:07:25,750 --> 00:07:34,860 So I'm going to have something called fast Fib. 100 00:07:34,860 --> 00:07:38,200 But first I'm going to have, let's look at what fast Fib 101 00:07:38,200 --> 00:07:41,450 does and then we'll come back to the next question. 102 00:07:41,450 --> 00:07:50,610 It takes the number whose Fibonacci I want plus a memo. 103 00:07:50,610 --> 00:07:56,330 And the memo will be a dictionary that maps me from a 104 00:07:56,330 --> 00:08:05,480 number to Fib of that number. 105 00:08:05,480 --> 00:08:09,680 So what I'm going to do, well, let's get rid of this print 106 00:08:09,680 --> 00:08:13,020 statement for now. 107 00:08:13,020 --> 00:08:19,260 I'm going to say, if n is not in memo. 108 00:08:19,260 --> 00:08:22,650 Remember the way dictionary works, this is the key. 109 00:08:22,650 --> 00:08:25,630 Is the key of a value. 110 00:08:25,630 --> 00:08:31,070 Then I'll call fast Fib recursively, with n minus 1 in 111 00:08:31,070 --> 00:08:38,410 memo, and n minus 2 in memo. 112 00:08:38,410 --> 00:08:42,660 Otherwise I'll return the memo. 113 00:08:42,660 --> 00:08:44,690 Well, let's look at it for a second. 114 00:08:44,690 --> 00:08:47,310 This is the basic idea. 115 00:08:47,310 --> 00:08:51,290 But do we actually believe this is going to work? 116 00:08:51,290 --> 00:08:53,920 And, again, I want you to look at this and think about what's 117 00:08:53,920 --> 00:08:56,310 going to happen here. 118 00:08:56,310 --> 00:09:02,270 Before we do that, or as you do that, let's look at Fib 1. 119 00:09:02,270 --> 00:09:07,780 The key thing to notice about Fib 1 is that it has the same 120 00:09:07,780 --> 00:09:12,900 specification as Fib. 121 00:09:12,900 --> 00:09:16,910 Because when somebody calls Fibonacci, they shouldn't 122 00:09:16,910 --> 00:09:19,050 worry about memos. 123 00:09:19,050 --> 00:09:21,110 And how I'd implemented it. 124 00:09:21,110 --> 00:09:23,780 That has to be under the covers. 125 00:09:23,780 --> 00:09:25,650 So I don't want them to have to call 126 00:09:25,650 --> 00:09:27,530 something with two arguments. 127 00:09:27,530 --> 00:09:30,540 The integer and the memo. 128 00:09:30,540 --> 00:09:37,200 So I'll create Fib 1, which has the same arguments as Fib. 129 00:09:37,200 --> 00:09:43,790 The first thing it does is it initializes the memo. 130 00:09:43,790 --> 00:09:47,800 And initializes it by saying, if I get 0 I -- whoops. 131 00:09:47,800 --> 00:09:50,410 Aha. 132 00:09:50,410 --> 00:09:56,250 Let's be careful here. 133 00:09:56,250 --> 00:09:58,540 If I get 0 I return 1. 134 00:09:58,540 --> 00:10:00,410 I get 1, I return 1. 135 00:10:00,410 --> 00:10:06,510 So I put two things in the memo already. 136 00:10:06,510 --> 00:10:10,360 And then I'll call fast Fib and it returns 137 00:10:10,360 --> 00:10:15,800 the result it has. 138 00:10:15,800 --> 00:10:18,370 So you see the basic idea. 139 00:10:18,370 --> 00:10:22,450 I take something with the same parameters as the original. 140 00:10:22,450 --> 00:10:23,560 Add this memo. 141 00:10:23,560 --> 00:10:26,020 Give it some initial values. 142 00:10:26,020 --> 00:10:30,230 And then call. 143 00:10:30,230 --> 00:10:32,180 So now what do we think? 144 00:10:32,180 --> 00:10:38,190 Is this going to work? 145 00:10:38,190 --> 00:10:42,610 Or is there an issue here? 146 00:10:42,610 --> 00:10:43,740 What do you think? 147 00:10:43,740 --> 00:10:45,770 Think it through. 148 00:10:45,770 --> 00:10:51,400 If it's not in the memo, I'll compute its value and put it 149 00:10:51,400 --> 00:10:53,110 in the memo. 150 00:10:53,110 --> 00:10:55,560 And then I'll return it. 151 00:10:55,560 --> 00:10:56,990 OK? 152 00:10:56,990 --> 00:11:01,450 If it was already there, I just look it up. 153 00:11:01,450 --> 00:11:04,050 That make sense to everybody? 154 00:11:04,050 --> 00:11:20,720 Let's see what happens if we run it. 155 00:11:20,720 --> 00:11:23,045 Well, actually, let's turn the print statement on, since 156 00:11:23,045 --> 00:11:32,400 we're doing it with a small value here. 157 00:11:32,400 --> 00:11:38,060 So what we've seen is I've run it twice here. 158 00:11:38,060 --> 00:11:43,050 When I ran it up here, with the old Fib, and we printed 159 00:11:43,050 --> 00:11:47,240 the result, and I ran it with Fib 1 down here. 160 00:11:47,240 --> 00:11:52,780 The good news is we got 13 both times. 161 00:11:52,780 --> 00:11:58,470 The even better news is that instead of 25 calls, it was 162 00:11:58,470 --> 00:12:05,480 only 11 calls. 163 00:12:05,480 --> 00:12:10,470 So it's a big improvement. 164 00:12:10,470 --> 00:12:13,560 Let's see what happens, just to get an idea of how big the 165 00:12:13,560 --> 00:12:15,460 improvement is. 166 00:12:15,460 --> 00:12:23,300 I'm going to take out the two print statements. 167 00:12:23,300 --> 00:12:39,340 And let's try it with a bigger number. 168 00:12:39,340 --> 00:12:42,230 It's going to take a little bit. 169 00:12:42,230 --> 00:12:46,630 Well, look at this difference. 170 00:12:46,630 --> 00:12:54,890 It's 2,692,537 versus 59. 171 00:12:54,890 --> 00:13:00,660 That's a pretty darn big difference. 172 00:13:00,660 --> 00:13:03,290 And I won't ask you to check whether it 173 00:13:03,290 --> 00:13:05,470 got the right answer. 174 00:13:05,470 --> 00:13:07,600 At least, not in your heads. 175 00:13:07,600 --> 00:13:11,940 So you can see, and this is an important thing we look at, is 176 00:13:11,940 --> 00:13:14,850 that as we look at growth, it didn't look like it mattered a 177 00:13:14,850 --> 00:13:15,810 lot with 6. 178 00:13:15,810 --> 00:13:17,670 Because it was one small number to one 179 00:13:17,670 --> 00:13:19,340 slightly smaller number. 180 00:13:19,340 --> 00:13:24,140 But this thing grows exponentially. 181 00:13:24,140 --> 00:13:26,640 It's a little bit complicated exactly how. 182 00:13:26,640 --> 00:13:31,700 But you can see as I go up to 30 I get a pretty big number. 183 00:13:31,700 --> 00:13:35,120 And 59 is a pretty small number. 184 00:13:35,120 --> 00:13:39,550 So we see that the memoization here is buying me 185 00:13:39,550 --> 00:13:43,070 a tremendous advantage. 186 00:13:43,070 --> 00:13:46,880 And this is what lies at the heart of this very general 187 00:13:46,880 --> 00:13:50,640 technique called dynamic programming. 188 00:13:50,640 --> 00:13:54,420 And in fact, it lies at the heart of a lot of useful 189 00:13:54,420 --> 00:14:00,190 computational techniques where we save results. 190 00:14:00,190 --> 00:14:03,790 So if you think about the way something like, say, Mapquest 191 00:14:03,790 --> 00:14:07,710 works, and last week in recitation you looked at the 192 00:14:07,710 --> 00:14:11,390 fact that shortest path is exponential. 193 00:14:11,390 --> 00:14:16,030 Well, what it does is it saves a lot of paths. 194 00:14:16,030 --> 00:14:18,440 It kind of knows people are going to ask how do you get 195 00:14:18,440 --> 00:14:21,730 from Boston to New York City. 196 00:14:21,730 --> 00:14:24,530 And it may have saved that. 197 00:14:24,530 --> 00:14:26,980 And if you're going from Boston to someplace else where 198 00:14:26,980 --> 00:14:30,100 New York just happens to be on the way, it doesn't have to 199 00:14:30,100 --> 00:14:32,600 recompute that part of it. 200 00:14:32,600 --> 00:14:36,790 So it's saved a lot of things and squirreled them away. 201 00:14:36,790 --> 00:14:38,560 And that's essentially what we're doing here. 202 00:14:38,560 --> 00:14:41,320 Here we're doing it as part of one algorithm. 203 00:14:41,320 --> 00:14:44,400 There, they're just storing a database of previously solved 204 00:14:44,400 --> 00:14:51,510 problems. And relying on something called table lookup, 205 00:14:51,510 --> 00:14:57,860 Of which memoization is a special case. 206 00:14:57,860 --> 00:14:59,980 But table lookup is very common. 207 00:14:59,980 --> 00:15:01,900 When you do something complicated you save the 208 00:15:01,900 --> 00:15:06,240 answers and then you go get it later. 209 00:15:06,240 --> 00:15:13,330 I should add that in some sense this is a phony 210 00:15:13,330 --> 00:15:15,890 straw-man Fibonacci. 211 00:15:15,890 --> 00:15:18,710 Nobody in their right mind actually implements a 212 00:15:18,710 --> 00:15:22,020 recursive Fibonacci the way I did it originally. 213 00:15:22,020 --> 00:15:24,690 Because the right way to do it is iteratively. 214 00:15:24,690 --> 00:15:27,290 And the right way to do it is not starting at the top, it's 215 00:15:27,290 --> 00:15:29,510 starting at the bottom. 216 00:15:29,510 --> 00:15:31,400 And so you can piece it together that way. 217 00:15:31,400 --> 00:15:34,930 But don't worry about it, it's not, I'm just using it because 218 00:15:34,930 --> 00:15:38,340 it's a simpler example than the one I really want to get 219 00:15:38,340 --> 00:15:41,280 to, which is knapsack. 220 00:15:41,280 --> 00:15:44,600 OK, people get this? 221 00:15:44,600 --> 00:15:50,560 And see the basic idea and why it's wonderful? 222 00:15:50,560 --> 00:15:52,480 Alright. 223 00:15:52,480 --> 00:15:55,910 Now, when we talked about optimization problems in 224 00:15:55,910 --> 00:15:59,080 dynamic programming, I said there were two 225 00:15:59,080 --> 00:16:02,710 things to look for. 226 00:16:02,710 --> 00:16:06,790 One was overlapping sub-problems. And the other 227 00:16:06,790 --> 00:16:17,770 one was optimal substructure. 228 00:16:17,770 --> 00:16:22,700 The notion here is that you can get a globally optimal 229 00:16:22,700 --> 00:16:56,190 solution from locally optimal solutions to sub-problems. 230 00:16:56,190 --> 00:17:04,900 This is not true of all problems. But as we'll see, 231 00:17:04,900 --> 00:17:08,690 it's true of a lot of problems. And when you have an 232 00:17:08,690 --> 00:17:15,320 optimal substructure and the local solutions overlap, 233 00:17:15,320 --> 00:17:19,030 that's when you can bring dynamic programming to bear. 234 00:17:19,030 --> 00:17:21,980 So when you're trying to think about is this a problem that I 235 00:17:21,980 --> 00:17:24,990 can solve with dynamic programming, these are the two 236 00:17:24,990 --> 00:17:29,480 questions you ask. 237 00:17:29,480 --> 00:17:33,760 Let's now go back and instantiate these ideas for 238 00:17:33,760 --> 00:17:35,870 the knapsack problem we looked at last time. 239 00:17:35,870 --> 00:17:42,940 In particular, for the 0-1 knapsack problem. 240 00:17:42,940 --> 00:17:47,880 So, we have a collection of objects. 241 00:17:47,880 --> 00:17:50,900 We'll call it a. 242 00:17:50,900 --> 00:17:54,210 And for each object in 0, we have a value. 243 00:17:54,210 --> 00:17:57,850 In a, we have a value. 244 00:17:57,850 --> 00:18:02,310 And now we want to find the subset of a that has the 245 00:18:02,310 --> 00:18:06,290 maximum value, subject to the weight constraint. 246 00:18:06,290 --> 00:18:09,700 I'm just repeating the problem. 247 00:18:09,700 --> 00:18:11,600 Now, what we saw last time is there's 248 00:18:11,600 --> 00:18:15,230 a brute force solution. 249 00:18:15,230 --> 00:18:18,610 As you have discovered in recent problem set, it is 250 00:18:18,610 --> 00:18:23,530 possible to construct all subsets of a set. 251 00:18:23,530 --> 00:18:28,530 And so you could construct all subsets, check that the weight 252 00:18:28,530 --> 00:18:31,290 is less than the weight of the knapsack, and then choose the 253 00:18:31,290 --> 00:18:33,790 subset with the maximum value. 254 00:18:33,790 --> 00:18:36,540 Or a subset with the maximum value, there may be more than 255 00:18:36,540 --> 00:18:40,910 one, and you're done. 256 00:18:40,910 --> 00:18:49,810 On the other hand, we've seen that if the size of a is n, 257 00:18:49,810 --> 00:18:54,360 that's to say, we have n elements to choose from, then 258 00:18:54,360 --> 00:18:58,230 the number of possible subsets is 2 to the n. 259 00:18:58,230 --> 00:19:00,190 Remember, we saw that last time looking 260 00:19:00,190 --> 00:19:03,670 at the binary numbers. 261 00:19:03,670 --> 00:19:07,180 2 to the n is a big number. 262 00:19:07,180 --> 00:19:09,840 And maybe we don't have to consider them all, because we 263 00:19:09,840 --> 00:19:12,660 can say, oh this one is going to be way too big. 264 00:19:12,660 --> 00:19:15,510 It's going to weigh too much, we don't need to look at it. 265 00:19:15,510 --> 00:19:19,700 But it'll still be order 2 to the n. 266 00:19:19,700 --> 00:19:25,220 If n is something like 50, not a big number, 2 to the 50 is a 267 00:19:25,220 --> 00:19:31,550 huge number. 268 00:19:31,550 --> 00:19:34,550 So let's ask, is there an optimal 269 00:19:34,550 --> 00:19:36,960 substructure to this problem. 270 00:19:36,960 --> 00:19:43,160 That would let us tackle it with dynamic programming. 271 00:19:43,160 --> 00:19:47,470 And we're going to do this initially by looking at a 272 00:19:47,470 --> 00:19:52,700 straightforward implementation based upon what's called the 273 00:19:52,700 --> 00:20:01,700 decision tree. 274 00:20:01,700 --> 00:20:05,210 This is a very important concept, and we'll see a lot 275 00:20:05,210 --> 00:20:10,570 of algorithms essentially implement decision trees. 276 00:20:10,570 --> 00:20:12,880 Let's look at an example. 277 00:20:12,880 --> 00:20:16,470 Let's assume that the weights, and I'll try a really small 278 00:20:16,470 --> 00:20:24,980 example to start with, are 5, 3 and 2, and the values, 279 00:20:24,980 --> 00:20:31,940 corresponding values, are 9, 7 and 8. 280 00:20:31,940 --> 00:20:39,460 And the maximum, we'll say, is 5. 281 00:20:39,460 --> 00:20:45,440 So what we do is, we start by considering for each item 282 00:20:45,440 --> 00:20:47,890 whether to take it or not. 283 00:20:47,890 --> 00:20:50,850 For reasons that will become apparent when we implement it 284 00:20:50,850 --> 00:20:55,620 in code, I'm going to start at the back. 285 00:20:55,620 --> 00:21:01,350 The last element in the list. And what I'm going to use is 286 00:21:01,350 --> 00:21:05,780 the index of that element to keep track of where I am. 287 00:21:05,780 --> 00:21:08,450 So I'm not going to worry whether this item is a vase or 288 00:21:08,450 --> 00:21:10,530 a watch or painting. 289 00:21:10,530 --> 00:21:13,130 I'm just going to say it's the n'th element. 290 00:21:13,130 --> 00:21:19,700 Where n'th is somewhere between 0 and 2 in this case. 291 00:21:19,700 --> 00:21:26,130 And then we'll construct our tree as follows: each node, 292 00:21:26,130 --> 00:21:28,040 well, let me put an example here. 293 00:21:28,040 --> 00:21:40,900 The first node will be the to-pull 2, 5 and 0. 294 00:21:40,900 --> 00:21:46,850 Standing for, let me make sure I get this in the right order, 295 00:21:46,850 --> 00:21:59,540 well, the index which is 2, the last element in this case, 296 00:21:59,540 --> 00:22:03,670 so that's the index. 297 00:22:03,670 --> 00:22:09,120 This is the weight still available. 298 00:22:09,120 --> 00:22:11,000 If you see that in the shadow. 299 00:22:11,000 --> 00:22:17,650 And this is the value currently obtained. 300 00:22:17,650 --> 00:22:21,260 So I haven't included anything. 301 00:22:21,260 --> 00:22:23,560 Means I have all 5 pounds left. 302 00:22:23,560 --> 00:22:27,010 But I don't have anything of value. 303 00:22:27,010 --> 00:22:32,690 Now, the decision tree, if I branch left, 304 00:22:32,690 --> 00:22:34,150 it's a binary tree. 305 00:22:34,150 --> 00:22:42,280 This is going to be don't take. 306 00:22:42,280 --> 00:22:48,790 So I'm not going to take the item with an index of 2. 307 00:22:48,790 --> 00:22:58,700 So that means this node will have an index of 1. 308 00:22:58,700 --> 00:23:05,220 Next item to be considered, I still have 5 pounds available. 309 00:23:05,220 --> 00:23:14,830 And I have 0 value. 310 00:23:14,830 --> 00:23:25,310 To be systematic, I'm going to build this tree depth-first 311 00:23:25,310 --> 00:23:36,680 left-first. At each node, I'm going to go left until I can't 312 00:23:36,680 --> 00:23:41,340 go any further. 313 00:23:41,340 --> 00:23:46,340 So we'll take another don't-take branch here. 314 00:23:46,340 --> 00:23:50,590 And what is this node going to look like? 315 00:23:50,590 --> 00:23:50,970 Pardon? 316 00:23:50,970 --> 00:23:53,200 STUDENT: [INAUDIBLE] 317 00:23:53,200 --> 00:23:57,680 PROFESSOR: 0, 5, 0. 318 00:23:57,680 --> 00:23:59,980 And then we'll go one more. 319 00:23:59,980 --> 00:24:03,430 And I'll just put a minus indicating I'm done. 320 00:24:03,430 --> 00:24:07,330 I can't look below that. 321 00:24:07,330 --> 00:24:13,550 I still have five pounds left, and I still have zero value. 322 00:24:13,550 --> 00:24:34,290 The next thing I'm going to do is backtrack. 323 00:24:34,290 --> 00:24:36,720 That is to say, I'm going to go back to a node 324 00:24:36,720 --> 00:24:38,620 I've already visited. 325 00:24:38,620 --> 00:24:43,050 Go up the tree 1. 326 00:24:43,050 --> 00:24:48,740 And now, of course, the only place to go is right. 327 00:24:48,740 --> 00:24:50,770 And now I get to include something. 328 00:24:50,770 --> 00:24:54,750 Yeah. 329 00:24:54,750 --> 00:24:57,950 And what does this node look like? 330 00:24:57,950 --> 00:25:04,710 Well, I'll give you a hint, it starts with a minus. 331 00:25:04,710 --> 00:25:05,540 What next? 332 00:25:05,540 --> 00:25:08,010 STUDENT: [INAUDIBLE] 333 00:25:08,010 --> 00:25:09,730 PROFESSOR: Pardon. 334 00:25:09,730 --> 00:25:10,710 0. 335 00:25:10,710 --> 00:25:11,300 And? 336 00:25:11,300 --> 00:25:12,660 STUDENT: [INAUDIBLE] 337 00:25:12,660 --> 00:25:14,170 PROFESSOR: Pardon? 338 00:25:14,170 --> 00:25:17,050 5. 339 00:25:17,050 --> 00:25:18,750 Alright. 340 00:25:18,750 --> 00:25:24,430 So far, this looks like the winner. 341 00:25:24,430 --> 00:25:26,780 But, we'd better keep going. 342 00:25:26,780 --> 00:25:28,790 We backtrack to here again. 343 00:25:28,790 --> 00:25:32,620 There's nothing useful to do. 344 00:25:32,620 --> 00:25:35,530 We backtrack to here. 345 00:25:35,530 --> 00:25:42,620 And we ask, what do we get with this node? 346 00:25:42,620 --> 00:25:47,230 0, 3. 347 00:25:47,230 --> 00:25:47,740 Somebody? 348 00:25:47,740 --> 00:25:49,380 STUDENT: [INAUDIBLE] 349 00:25:49,380 --> 00:25:52,120 PROFESSOR: Louder. 350 00:25:52,120 --> 00:25:54,570 2. 351 00:25:54,570 --> 00:25:55,100 And then? 352 00:25:55,100 --> 00:26:01,210 STUDENT: [INAUDIBLE] 353 00:26:01,210 --> 00:26:04,940 PROFESSOR: There's a value here, what's this value? 354 00:26:04,940 --> 00:26:09,680 I've included item number 1, which has a value of 7. 355 00:26:09,680 --> 00:26:17,010 Right? 356 00:26:17,010 --> 00:26:23,370 So now this looks like the winner. 357 00:26:23,370 --> 00:26:23,670 Pardon? 358 00:26:23,670 --> 00:26:27,710 STUDENT: [INAUDIBLE] 359 00:26:27,710 --> 00:26:28,230 PROFESSOR: This one? 360 00:26:28,230 --> 00:26:31,580 STUDENT: Yeah. 361 00:26:31,580 --> 00:26:31,900 [INAUDIBLE] 362 00:26:31,900 --> 00:26:36,020 PROFESSOR: Remember, I'm working from the back. 363 00:26:36,020 --> 00:26:38,260 So it shouldn't be 9. 364 00:26:38,260 --> 00:26:40,010 Should be what? 365 00:26:40,010 --> 00:26:42,760 Item 0, oh, you're right, items 0 is 9. 366 00:26:42,760 --> 00:26:45,040 Thank you. 367 00:26:45,040 --> 00:26:48,290 Right you are. 368 00:26:48,290 --> 00:26:52,670 Still looks like the winner. 369 00:26:52,670 --> 00:26:53,480 I can't hear you. 370 00:26:53,480 --> 00:27:03,050 STUDENT: [INAUDIBLE] 371 00:27:03,050 --> 00:27:06,780 PROFESSOR: Let's be careful about this. 372 00:27:06,780 --> 00:27:10,410 I'm glad people are watching. 373 00:27:10,410 --> 00:27:11,900 So we're here. 374 00:27:11,900 --> 00:27:17,710 And now we've got 5 pounds available. 375 00:27:17,710 --> 00:27:18,330 That's good. 376 00:27:18,330 --> 00:27:22,330 And we're considering item number 0. 377 00:27:22,330 --> 00:27:26,700 Which happens to weigh 5 pounds. 378 00:27:26,700 --> 00:27:29,760 So that's a good thing. 379 00:27:29,760 --> 00:27:35,130 So, it's the last item to consider. 380 00:27:35,130 --> 00:27:40,730 If we include it, we'll have nothing left. 381 00:27:40,730 --> 00:27:43,580 Because we had 5 and we're using all 5. 382 00:27:43,580 --> 00:27:46,070 So that will be 0. 383 00:27:46,070 --> 00:27:53,500 And its value is 9. 384 00:27:53,500 --> 00:27:55,990 So we put one item in the backpack and we've 385 00:27:55,990 --> 00:27:58,290 got a value of 9. 386 00:27:58,290 --> 00:28:01,660 Anyone have a problem with that? 387 00:28:01,660 --> 00:28:07,740 So far, so good? 388 00:28:07,740 --> 00:28:09,710 Now we've backed up to here. 389 00:28:09,710 --> 00:28:13,000 We're considering item 1 and we're trying to ask whether we 390 00:28:13,000 --> 00:28:14,880 can put it in. 391 00:28:14,880 --> 00:28:17,430 Item 1 has a weight of 3. 392 00:28:17,430 --> 00:28:22,570 So it would fit. 393 00:28:22,570 --> 00:28:26,280 And if we use it, we have 2 pounds left. 394 00:28:26,280 --> 00:28:29,420 And item 1 has a value of 7. 395 00:28:29,420 --> 00:28:33,450 So if we did this, we'd have a value of 7. 396 00:28:33,450 --> 00:28:36,730 But we're not done yet, right? 397 00:28:36,730 --> 00:28:40,320 We still have some things to consider. 398 00:28:40,320 --> 00:28:46,790 Well, we could consider not putting in item 0. 399 00:28:46,790 --> 00:28:48,620 That makes perfect sense. 400 00:28:48,620 --> 00:28:56,520 And we're back to where we're there, minus 2 and 7. 401 00:28:56,520 --> 00:29:02,390 And now let's ask the question how, about putting in item 0. 402 00:29:02,390 --> 00:29:05,700 Well, we can't. 403 00:29:05,700 --> 00:29:09,200 Because it would weigh 5 pounds, I only have 2 left. 404 00:29:09,200 --> 00:29:17,850 So there is no right branch to this one. 405 00:29:17,850 --> 00:29:20,130 So I'm making whatever decisions I can 406 00:29:20,130 --> 00:29:24,170 make along the way. 407 00:29:24,170 --> 00:29:26,940 Let's back up to here. 408 00:29:26,940 --> 00:29:31,430 And now we're going to ask about taking item 2. 409 00:29:31,430 --> 00:29:37,510 If we take item 2, then, well, the index after that will of 410 00:29:37,510 --> 00:29:39,610 course be 1. 411 00:29:39,610 --> 00:29:45,990 And the available weight will be 3. 412 00:29:45,990 --> 00:29:53,780 And the value will be 8, right? 413 00:29:53,780 --> 00:29:59,300 Alright, now we say, can I take item 1. 414 00:29:59,300 --> 00:29:59,470 Yeah. 415 00:29:59,470 --> 00:30:01,070 I can. 416 00:30:01,070 --> 00:30:03,960 It only weighs 3 and I happen to have 3 left. 417 00:30:03,960 --> 00:30:07,300 So that's good. 418 00:30:07,300 --> 00:30:08,300 Don't take it, right. 419 00:30:08,300 --> 00:30:08,770 Sorry. 420 00:30:08,770 --> 00:30:10,530 This is the don't take branch. 421 00:30:10,530 --> 00:30:17,080 So I go to 0, 3, 8. 422 00:30:17,080 --> 00:30:20,560 And then I can do another don't take branch. 423 00:30:20,560 --> 00:30:28,870 And this gets me to what, minus 3, 8. 424 00:30:28,870 --> 00:30:33,090 I'll now back up. 425 00:30:33,090 --> 00:30:37,650 And I'll say, alright, suppose I do take item 0, well I can't 426 00:30:37,650 --> 00:30:39,500 take item 0, right? 427 00:30:39,500 --> 00:30:41,710 Weighs too much. 428 00:30:41,710 --> 00:30:46,570 So, don't have that branch. 429 00:30:46,570 --> 00:30:49,060 Back up to here. 430 00:30:49,060 --> 00:30:51,840 Alright, can I take item 1? 431 00:30:51,840 --> 00:30:55,470 Yes, I can. 432 00:30:55,470 --> 00:30:57,200 And that gives me what? 433 00:30:57,200 --> 00:31:03,540 STUDENT: [INAUDIBLE] 434 00:31:03,540 --> 00:31:07,290 PROFESSOR: We have a winner. 435 00:31:07,290 --> 00:31:10,770 So it's kind of tedious, but it's important 436 00:31:10,770 --> 00:31:13,790 to see that it works. 437 00:31:13,790 --> 00:31:15,620 It's systematic. 438 00:31:15,620 --> 00:31:21,660 I have a way of exploring the possible solutions. 439 00:31:21,660 --> 00:31:29,230 And at the end I choose the winner. 440 00:31:29,230 --> 00:31:34,950 What's the complexity of this decision tree solution? 441 00:31:34,950 --> 00:31:39,730 Well, in the worst case, we're enumerating every possibility 442 00:31:39,730 --> 00:31:41,610 of in and out. 443 00:31:41,610 --> 00:31:44,690 Now I've shortened it a little bit by saying, ah, we've run 444 00:31:44,690 --> 00:31:46,430 out of weight, we're OK. 445 00:31:46,430 --> 00:31:51,270 But effectively it is, as we saw before, exponential. 446 00:31:51,270 --> 00:31:55,090 2 to the n, every value in the bit vector we looked at last 447 00:31:55,090 --> 00:31:57,320 time is either 0 or 1. 448 00:31:57,320 --> 00:32:04,840 So it's a binary number of n bits, 2 to the n. 449 00:32:04,840 --> 00:32:06,190 Let's look at a straightforward 450 00:32:06,190 --> 00:32:11,550 implementation of this. 451 00:32:11,550 --> 00:32:15,780 I'll get rid of Fibonacci here, we don't want to bother 452 00:32:15,780 --> 00:32:19,500 looking at that again. 453 00:32:19,500 --> 00:32:22,360 Hold on a second until I comment this out, yes. 454 00:32:22,360 --> 00:32:28,230 STUDENT: [INAUDIBLE] 455 00:32:28,230 --> 00:32:29,360 PROFESSOR: Yeah. 456 00:32:29,360 --> 00:32:31,500 There's a branch we could finish here, but since we're 457 00:32:31,500 --> 00:32:35,240 out of weight we sort of know we're going to be done. 458 00:32:35,240 --> 00:32:38,020 So we could complete it. 459 00:32:38,020 --> 00:32:44,950 But it's not very interesting. 460 00:32:44,950 --> 00:32:54,360 But yes, we probably should have done that. 461 00:32:54,360 --> 00:33:05,990 So let's look at an implementation here. 462 00:33:05,990 --> 00:33:06,690 Whoops. 463 00:33:06,690 --> 00:33:16,110 You had these in the handout, by the way. 464 00:33:16,110 --> 00:33:20,820 So here's max val. 465 00:33:20,820 --> 00:33:24,290 It takes four arguments. 466 00:33:24,290 --> 00:33:27,920 The weight, w, and v, these are the two 467 00:33:27,920 --> 00:33:30,270 vectors we've seen here. 468 00:33:30,270 --> 00:33:33,790 Of the weights and the values. 469 00:33:33,790 --> 00:33:39,940 It takes i, which is in some sense the length of those 470 00:33:39,940 --> 00:33:45,330 vectors, minus 1, because of the way Python works. 471 00:33:45,330 --> 00:33:47,580 So that gives me my index. 472 00:33:47,580 --> 00:33:51,530 And the amount of weight available, a w, 473 00:33:51,530 --> 00:33:57,760 for available weight. 474 00:33:57,760 --> 00:34:03,770 So again, I put in this num calls, which you can ignore. 475 00:34:03,770 --> 00:34:09,300 First line says, if i is 0, that means I'm looking at the 476 00:34:09,300 --> 00:34:13,190 very last element. 477 00:34:13,190 --> 00:34:16,340 Then if the weight of i is less than the available 478 00:34:16,340 --> 00:34:24,580 weight, I can return the value of i. 479 00:34:24,580 --> 00:34:27,820 Otherwise it's 0. 480 00:34:27,820 --> 00:34:29,750 I've got one element to look at. 481 00:34:29,750 --> 00:34:31,860 I either put it in if I can. 482 00:34:31,860 --> 00:34:34,830 If I can't, I don't. 483 00:34:34,830 --> 00:34:42,430 Alright, so if I'm at the end of the chain, that's my value. 484 00:34:42,430 --> 00:34:44,170 In either event, if I'm looking at the 485 00:34:44,170 --> 00:34:47,060 last element, I return. 486 00:34:47,060 --> 00:34:52,790 The next line says alright, suppose I don't, I'm not at 487 00:34:52,790 --> 00:34:54,810 the last element. 488 00:34:54,810 --> 00:34:58,110 Suppose I don't include it. 489 00:34:58,110 --> 00:35:04,330 Then the maximum value I can get is the maximum value of 490 00:35:04,330 --> 00:35:05,420 what I had. 491 00:35:05,420 --> 00:35:09,770 But with index i minus 1. 492 00:35:09,770 --> 00:35:16,220 So that's the don't-take branch. 493 00:35:16,220 --> 00:35:20,450 As we've seen systematically in the don't-takes, the only 494 00:35:20,450 --> 00:35:28,450 thing that gets changed is the index. 495 00:35:28,450 --> 00:35:34,970 The next line says if the weight of i is greater than a 496 00:35:34,970 --> 00:35:39,710 w, well then I know I can't put it in. 497 00:35:39,710 --> 00:35:41,840 So I might as well return the without i 498 00:35:41,840 --> 00:35:46,370 value I just computed. 499 00:35:46,370 --> 00:35:51,220 Otherwise, let's see what I get with i. 500 00:35:51,220 --> 00:35:58,140 And so the value of with i will be the value of i plus 501 00:35:58,140 --> 00:36:05,080 whatever I can get using the remaining items and 502 00:36:05,080 --> 00:36:08,460 decrementing the weight by the weight of i. 503 00:36:08,460 --> 00:36:10,620 So that's exactly what we see as we go 504 00:36:10,620 --> 00:36:13,750 down the right branches. 505 00:36:13,750 --> 00:36:17,600 I look at the rest of the list, but 506 00:36:17,600 --> 00:36:19,970 I've changed the value. 507 00:36:19,970 --> 00:36:25,950 And I've changed the available weight. 508 00:36:25,950 --> 00:36:29,330 And then when I get to the very end, I'm going to return 509 00:36:29,330 --> 00:36:35,100 the bigger of with i and without i. 510 00:36:35,100 --> 00:36:37,730 I've computed the value if I include i. 511 00:36:37,730 --> 00:36:40,580 I computed the value if I don't include i. 512 00:36:40,580 --> 00:36:47,170 And then I'm going to just return the bigger of the two. 513 00:36:47,170 --> 00:36:50,630 Little bit complicated, but it's basically just 514 00:36:50,630 --> 00:36:58,810 implementing this decision tree. 515 00:36:58,810 --> 00:37:02,340 Let's see what happens if we run it. 516 00:37:02,340 --> 00:37:05,350 Well, what I always do in anything like this is, the 517 00:37:05,350 --> 00:37:08,940 first thing I do is, I run it on something where I can 518 00:37:08,940 --> 00:37:13,130 actually compute the answer in my head. 519 00:37:13,130 --> 00:37:18,660 So I get a sense of whether or not I'm doing the right thing. 520 00:37:18,660 --> 00:37:26,820 So here's a little example. 521 00:37:26,820 --> 00:37:30,510 And I'm going to pause for a minute, before I run it, and 522 00:37:30,510 --> 00:37:33,160 ask each of you to compute in your head what you think the 523 00:37:33,160 --> 00:37:38,710 answer should be. 524 00:37:38,710 --> 00:37:40,460 In line with what I said about debugging. 525 00:37:40,460 --> 00:37:42,710 Always guess before you run your program what you think 526 00:37:42,710 --> 00:37:43,300 it's going to do. 527 00:37:43,300 --> 00:37:50,590 And I'm going to wait until somebody raises their hand and 528 00:37:50,590 --> 00:37:59,060 gives me an answer. 529 00:37:59,060 --> 00:37:59,580 Yes. 530 00:37:59,580 --> 00:38:01,250 STUDENT: 29? 531 00:38:01,250 --> 00:38:04,430 PROFESSOR: So we have a hypothesis that the answer 532 00:38:04,430 --> 00:38:10,050 should be 29. 533 00:38:10,050 --> 00:38:11,190 Ooh. 534 00:38:11,190 --> 00:38:14,550 That's bad. 535 00:38:14,550 --> 00:38:16,860 So as people in the back are answering these questions 536 00:38:16,860 --> 00:38:23,370 because they want to test my arm. 537 00:38:23,370 --> 00:38:25,320 The camera probably didn't catch that, but it was a 538 00:38:25,320 --> 00:38:29,450 perfect throw. 539 00:38:29,450 --> 00:38:32,760 Anyone else think it's 29? 540 00:38:32,760 --> 00:38:35,460 Anyone think it's not 29? 541 00:38:35,460 --> 00:38:37,710 What do you think it is? 542 00:38:37,710 --> 00:38:38,920 Pardon 543 00:38:38,920 --> 00:38:39,470 STUDENT: 62? 544 00:38:39,470 --> 00:38:40,630 PROFESSOR: 62. 545 00:38:40,630 --> 00:38:42,780 That would be impressive. 546 00:38:42,780 --> 00:38:44,520 62. 547 00:38:44,520 --> 00:38:46,330 Alright, well, we have a guess of 62. 548 00:38:46,330 --> 00:38:54,980 Well, let's run it and see. 549 00:38:54,980 --> 00:38:58,300 29 it is. 550 00:38:58,300 --> 00:39:02,440 And it made a total of 13 calls. 551 00:39:02,440 --> 00:39:08,900 It didn't do this little optimization I did over here. 552 00:39:08,900 --> 00:39:11,410 But it gives us our answer. 553 00:39:11,410 --> 00:39:16,760 So that's pretty good. 554 00:39:16,760 --> 00:39:23,000 Let's try a slightly larger example. 555 00:39:23,000 --> 00:39:25,240 So here I'm going to use the example we had 556 00:39:25,240 --> 00:39:28,390 in class last time. 557 00:39:28,390 --> 00:39:31,290 This was the burglar example where they had two copies of 558 00:39:31,290 --> 00:39:39,610 everything. 559 00:39:39,610 --> 00:39:47,730 Here, it gets a maximum value of 48 and 85 calls. 560 00:39:47,730 --> 00:39:51,640 So we see that I've doubled the size of the vector but 561 00:39:51,640 --> 00:39:56,470 I've much more than doubled the number of calls. 562 00:39:56,470 --> 00:39:59,340 This is one of these properties of this kind of 563 00:39:59,340 --> 00:40:01,360 exponential growth. 564 00:40:01,360 --> 00:40:05,050 Well, let's be really brave here. 565 00:40:05,050 --> 00:40:09,000 And let's try a really big vector. 566 00:40:09,000 --> 00:40:17,730 So this particular vector, you probably can't even see the 567 00:40:17,730 --> 00:40:20,460 whole thing on your screen. 568 00:40:20,460 --> 00:40:32,950 Well, it's got 40 items in it. 569 00:40:32,950 --> 00:40:40,250 Let's see what happens here. 570 00:40:40,250 --> 00:40:42,770 Alright, who wants to tell me what the answer is while this 571 00:40:42,770 --> 00:40:45,170 computes away? 572 00:40:45,170 --> 00:40:47,980 Nobody in their right mind. 573 00:40:47,980 --> 00:40:50,230 I can tell you the answer, but that's because I've cheated, 574 00:40:50,230 --> 00:40:52,890 run the program before. 575 00:40:52,890 --> 00:40:56,840 Alright, this is going to take a while. 576 00:40:56,840 --> 00:41:01,120 And why is it going to take a while? 577 00:41:01,120 --> 00:41:04,910 Actually it's not 40, I think these are, alright. 578 00:41:04,910 --> 00:41:07,540 So the answer is 75 and the number of 579 00:41:07,540 --> 00:41:14,730 calls is 1.7 million. 580 00:41:14,730 --> 00:41:15,390 Pardon? 581 00:41:15,390 --> 00:41:17,710 17 million. 582 00:41:17,710 --> 00:41:20,690 Computers are fast, fortunately. 583 00:41:20,690 --> 00:41:29,930 Well, that's a lot of calls. 584 00:41:29,930 --> 00:41:32,470 Let's try and figure out what's going on. 585 00:41:32,470 --> 00:41:35,090 But let's not try and figure out what's going on with this 586 00:41:35,090 --> 00:41:51,810 big, big example because we'll get really tired. 587 00:41:51,810 --> 00:41:52,230 Oh. 588 00:41:52,230 --> 00:41:56,230 Actually, before we do that, just for fun, what I want to 589 00:41:56,230 --> 00:42:01,740 do is write down for future reference, as we look at a 590 00:42:01,740 --> 00:42:09,270 fast one, that the answer is 75 and Eric, how many calls? 591 00:42:09,270 --> 00:42:15,100 240,000. 592 00:42:15,100 --> 00:42:16,310 Alright. 593 00:42:16,310 --> 00:42:22,050 We'll come back to those numbers. 594 00:42:22,050 --> 00:42:25,920 Let's look at it with a smaller example. 595 00:42:25,920 --> 00:42:32,040 We'll look at our example of 8. 596 00:42:32,040 --> 00:42:34,710 That we looked at before. 597 00:42:34,710 --> 00:42:37,480 And we'll turn on this print statement. 598 00:42:37,480 --> 00:42:51,540 Ooh, what was that? 599 00:42:51,540 --> 00:42:54,100 Notice that I'm only printing i and a w. 600 00:42:54,100 --> 00:42:56,280 Why is that? 601 00:42:56,280 --> 00:42:59,490 Because w and v are constant. 602 00:42:59,490 --> 00:43:10,480 I don't want you to print them over and over again. 603 00:43:10,480 --> 00:43:12,440 No, I'd better call it. 604 00:43:12,440 --> 00:43:17,110 That would be a good thing to do, right? 605 00:43:17,110 --> 00:43:22,810 So let's see. 606 00:43:22,810 --> 00:43:41,040 We'll call it with this one. 607 00:43:41,040 --> 00:43:42,870 So it's printing a lot. 608 00:43:42,870 --> 00:43:46,000 It'll print, I think, 85 calls. 609 00:43:46,000 --> 00:43:49,210 And the thing you should notice here is that it's doing 610 00:43:49,210 --> 00:43:52,120 a lot of the same things over and over again. 611 00:43:52,120 --> 00:43:56,340 So, for example, we'll see 2, 1 here. 612 00:43:56,340 --> 00:43:58,530 And 2, 1 here. 613 00:43:58,530 --> 00:44:03,320 And 2, 1 here. 614 00:44:03,320 --> 00:44:06,480 Just like Fibonacci, it's doing the same work over and 615 00:44:06,480 --> 00:44:11,710 over again. 616 00:44:11,710 --> 00:44:15,090 So what's the solution? 617 00:44:15,090 --> 00:44:17,890 Well, you won't be surprised to hear 618 00:44:17,890 --> 00:44:20,540 it's the same solution. 619 00:44:20,540 --> 00:44:26,500 So let's look at that code. 620 00:44:26,500 --> 00:44:45,240 So I'm going to do exactly the same trick we did before. 621 00:44:45,240 --> 00:44:56,710 I don't want b to the Fibonacci. 622 00:44:56,710 --> 00:45:01,310 I'm going to introduce a max val 0, which has exactly the 623 00:45:01,310 --> 00:45:03,870 same arguments as max val. 624 00:45:03,870 --> 00:45:09,240 Here I'll initiate the memo to be 0, or to be empty, rather, 625 00:45:09,240 --> 00:45:10,650 the dictionary. 626 00:45:10,650 --> 00:45:14,380 And then I'll call fast max val passing at this extra 627 00:45:14,380 --> 00:45:20,320 argument of the memo. 628 00:45:20,320 --> 00:45:24,560 So the first thing I'm going to do is, I'm going to try and 629 00:45:24,560 --> 00:45:28,230 return the value in the memo. 630 00:45:28,230 --> 00:45:30,470 This is a good thing to do, right? 631 00:45:30,470 --> 00:45:34,730 If it's not already there, what will happen? 632 00:45:34,730 --> 00:45:36,560 It will raise an exception. 633 00:45:36,560 --> 00:45:39,710 And I'll go to the except clause. 634 00:45:39,710 --> 00:45:44,950 So I'm using the Python try except to check whether or not 635 00:45:44,950 --> 00:45:47,600 the thing is in the memo or not. 636 00:45:47,600 --> 00:45:50,900 I try and return the value. 637 00:45:50,900 --> 00:45:52,270 If it's there, I return it. 638 00:45:52,270 --> 00:45:57,010 If not I'll get a key error. 639 00:45:57,010 --> 00:46:00,270 And what do I do if I get a key error? 640 00:46:00,270 --> 00:46:05,470 Well, now I go through code very much like what I did for 641 00:46:05,470 --> 00:46:09,150 max val in the first place. 642 00:46:09,150 --> 00:46:12,340 I check whether it's 0, et cetera, et cetera. 643 00:46:12,340 --> 00:46:18,340 It's exactly, really, the same, except before I return 644 00:46:18,340 --> 00:46:22,160 the value I would have returned I squirrel it away in 645 00:46:22,160 --> 00:46:28,690 my memo for future use. 646 00:46:28,690 --> 00:46:32,500 So it's the same structure as before. 647 00:46:32,500 --> 00:46:36,730 Except, before I return the value, I save it. 648 00:46:36,730 --> 00:46:47,610 So the next time I need it, I can look it up. 649 00:46:47,610 --> 00:46:54,190 Let's see if it works. 650 00:46:54,190 --> 00:47:28,135 Well, let's do a small example first. It's 651 00:47:28,135 --> 00:47:29,580 calling the old max val. 652 00:47:29,580 --> 00:47:30,660 With all those print statements. 653 00:47:30,660 --> 00:47:31,940 Sorry about that. 654 00:47:31,940 --> 00:47:35,190 But we'll let it run. 655 00:47:35,190 --> 00:47:39,800 Well, it's a little bit better. 656 00:47:39,800 --> 00:47:43,380 It got 85. 657 00:47:43,380 --> 00:47:46,550 Same answer, but 50 calls instead of 85. 658 00:47:46,550 --> 00:47:50,580 But let's try the big one. 659 00:47:50,580 --> 00:47:57,780 Because we're really brave. So here's where we have 30 660 00:47:57,780 --> 00:48:06,280 elements, and a maximum weight of 40. 661 00:48:06,280 --> 00:48:08,880 I'm not going to call the other max val here, because we 662 00:48:08,880 --> 00:48:11,070 know what happens when I do that. 663 00:48:11,070 --> 00:48:15,770 I've created my own little memo over there. 664 00:48:15,770 --> 00:48:18,450 And let's let it rip. 665 00:48:18,450 --> 00:48:20,980 Wow. 666 00:48:20,980 --> 00:48:23,350 Well, I got the same answer. 667 00:48:23,350 --> 00:48:24,910 That's a good thing. 668 00:48:24,910 --> 00:48:31,350 And instead of 17 million calls, I have 1800 calls. 669 00:48:31,350 --> 00:48:34,230 That's a huge improvement. 670 00:48:34,230 --> 00:48:38,610 And that's sort of the magic of dynamic programming. 671 00:48:38,610 --> 00:48:43,370 On Thursday we'll do a more careful analysis and try and 672 00:48:43,370 --> 00:48:47,630 understand how I could have accomplished this seemingly 673 00:48:47,630 --> 00:48:51,750 magical task of solving an exponential 674 00:48:51,750 --> 00:48:54,870 problem so really quickly.