1 00:00:00,000 --> 00:00:00,499 2 00:00:00,499 --> 00:00:03,144 The following content is provided under a Creative 3 00:00:03,144 --> 00:00:03,810 Commons license. 4 00:00:03,810 --> 00:00:05,518 Your support will help MIT OpenCourseWare 5 00:00:05,518 --> 00:00:09,930 continue to offer high-quality educational resources for free. 6 00:00:09,930 --> 00:00:12,540 To make a donation, or to view additional materials 7 00:00:12,540 --> 00:00:16,150 from hundreds of MIT courses, visit MIT OpenCourseWare 8 00:00:16,150 --> 00:00:21,310 at ocw.mit.edu. 9 00:00:21,310 --> 00:00:23,470 OK, so. 10 00:00:23,470 --> 00:00:25,050 Pretty full day again. 11 00:00:25,050 --> 00:00:30,730 I had last time introduced the Fourier matrix, 12 00:00:30,730 --> 00:00:33,330 the discrete Fourier transform. 13 00:00:33,330 --> 00:00:37,400 Well, more strictly, the discrete Fourier transform 14 00:00:37,400 --> 00:00:38,530 is usually this one. 15 00:00:38,530 --> 00:00:43,250 It takes the function values and produces the coefficients. 16 00:00:43,250 --> 00:00:47,830 And then I started with the coefficients, added back, 17 00:00:47,830 --> 00:00:51,080 added up the series to get the function values. 18 00:00:51,080 --> 00:00:53,620 So F or F inverse. 19 00:00:53,620 --> 00:00:56,430 So we didn't do examples yet. 20 00:00:56,430 --> 00:01:03,380 And one natural example is the discrete delta function that 21 00:01:03,380 --> 00:01:06,050 has a one in the zero position. 22 00:01:06,050 --> 00:01:07,540 That's easy to do. 23 00:01:07,540 --> 00:01:11,170 And then we should do also a shift, 24 00:01:11,170 --> 00:01:15,080 to see what's the effect, if you shift the function, what 25 00:01:15,080 --> 00:01:17,020 happens to the transform. 26 00:01:17,020 --> 00:01:19,780 That's an important rule, important for Fourier series 27 00:01:19,780 --> 00:01:21,890 and Fourier integrals, too. 28 00:01:21,890 --> 00:01:26,240 Because often you do that, and it's going to be a simple rule. 29 00:01:26,240 --> 00:01:31,490 If you shift the function, the transform does something nice. 30 00:01:31,490 --> 00:01:35,520 OK, and then I want to describe a little about the FFT 31 00:01:35,520 --> 00:01:38,940 and then start on the next section, convolutions. 32 00:01:38,940 --> 00:01:42,250 So that's fun and that's a big deal. 33 00:01:42,250 --> 00:01:47,340 OK, about reviews, I'll be here as usual today. 34 00:01:47,340 --> 00:01:52,690 I think maybe the 26th, just hours before Thanksgiving, 35 00:01:52,690 --> 00:01:56,230 we can give ourselves a holiday. 36 00:01:56,230 --> 00:02:01,120 So not next Wednesday but then certainly the Wednesday 37 00:02:01,120 --> 00:02:05,500 of the following week would be a sort of major quiz 38 00:02:05,500 --> 00:02:10,040 review in the review session. 39 00:02:10,040 --> 00:02:11,790 And in class. 40 00:02:11,790 --> 00:02:13,950 OK, ready to go? 41 00:02:13,950 --> 00:02:18,390 On this example gives us a chance 42 00:02:18,390 --> 00:02:21,480 just to remember what the matrix looks like, because we're just 43 00:02:21,480 --> 00:02:25,960 going to, if I multiply this inverse matrix by that vector 44 00:02:25,960 --> 00:02:27,865 it's just going to pick off the first column 45 00:02:27,865 --> 00:02:31,900 and it'll be totally easy so let's just do it. 46 00:02:31,900 --> 00:02:36,820 So if y is this one, I want to know about f. 47 00:02:36,820 --> 00:02:38,380 What are the Fourier coefficients 48 00:02:38,380 --> 00:02:44,020 of the delta function? 49 00:02:44,020 --> 00:02:45,700 Discrete delta function? 50 00:02:45,700 --> 00:02:49,560 OK, before I even do it, we got a pretty good idea 51 00:02:49,560 --> 00:02:51,230 what to expect. 52 00:02:51,230 --> 00:02:54,130 Because we remember what happened to the ordinary delta 53 00:02:54,130 --> 00:02:57,040 function, in continuous time. 54 00:02:57,040 --> 00:03:00,770 Or rather, I guess it was the periodic delta function. 55 00:03:00,770 --> 00:03:02,210 Do you remember the coefficients, 56 00:03:02,210 --> 00:03:05,460 what were the coefficients for the periodic delta function? 57 00:03:05,460 --> 00:03:07,370 You remember those? 58 00:03:07,370 --> 00:03:09,220 We took the integral from minus pi 59 00:03:09,220 --> 00:03:13,340 to pi of our function, which was delta(x). 60 00:03:13,340 --> 00:03:17,240 And then we had to remember to divide by 2pi, 61 00:03:17,240 --> 00:03:23,090 and you remember the coefficients are e^(-ikx). 62 00:03:23,090 --> 00:03:29,130 This is c_k in the periodic case, the 2pi periodic case. 63 00:03:29,130 --> 00:03:31,120 The function is the delta function, 64 00:03:31,120 --> 00:03:35,200 and you remember that if we want coefficient k we multiply 65 00:03:35,200 --> 00:03:37,730 by e^(-ikx). 66 00:03:37,730 --> 00:03:42,140 That's the thing that will pick out the e^(+ikx) term. 67 00:03:42,140 --> 00:03:47,100 And of course everybody knows what we get here, this delta, 68 00:03:47,100 --> 00:03:52,500 this spike at x=0, means we take the value of that function 69 00:03:52,500 --> 00:03:57,780 at zero, which is one, so we just get 1/(2pi). 70 00:03:57,780 --> 00:04:02,490 For all the Fourier coefficients of the delta function. 71 00:04:02,490 --> 00:04:05,660 The point being that they're all the same. 72 00:04:05,660 --> 00:04:08,140 That all frequencies are in the delta function 73 00:04:08,140 --> 00:04:09,350 to the same amount. 74 00:04:09,350 --> 00:04:12,220 I mean that's kind of nice. 75 00:04:12,220 --> 00:04:15,040 We created the delta function for other reasons, 76 00:04:15,040 --> 00:04:21,190 but then here in Fourier space it's just clean as could be. 77 00:04:21,190 --> 00:04:23,470 And we'll expect something here, too. 78 00:04:23,470 --> 00:04:25,960 You remember what F inverse is? 79 00:04:25,960 --> 00:04:31,530 F inverse is 1/N, instead of 1/2pi, and then 80 00:04:31,530 --> 00:04:38,190 the entries of F inverse come from F bar, the conjugate. 81 00:04:38,190 --> 00:04:41,630 So it's just one, one, one, minus-- Well, 82 00:04:41,630 --> 00:04:44,360 I've made it four by four here. 83 00:04:44,360 --> 00:04:50,970 This is minus omega-- No, it isn't minus omega. 84 00:04:50,970 --> 00:04:58,620 It's it's omega bar, which is minus i, in this case. 85 00:04:58,620 --> 00:05:02,390 Omega bar, so it's minus i, that's omega bar. 86 00:05:02,390 --> 00:05:05,950 And the next one would be omega bar squared, and cubed, 87 00:05:05,950 --> 00:05:09,530 and so on. 88 00:05:09,530 --> 00:05:12,140 All the way up to the ninth power. 89 00:05:12,140 --> 00:05:14,710 But we're multiplying by one, zero, zero, 90 00:05:14,710 --> 00:05:19,580 so none of that matters. 91 00:05:19,580 --> 00:05:24,400 What's the answer? 92 00:05:24,400 --> 00:05:27,640 I'm doing this discrete Fourier transform, 93 00:05:27,640 --> 00:05:31,620 so I'm multiplying by the matrix with the complex conjugate 94 00:05:31,620 --> 00:05:32,720 guys. 95 00:05:32,720 --> 00:05:34,720 But I'm multiplying by that simple thing 96 00:05:34,720 --> 00:05:37,840 so it's just going to pick out the zeroth column. 97 00:05:37,840 --> 00:05:40,440 In other words, constant. 98 00:05:40,440 --> 00:05:44,270 All the discrete Fourier coefficients 99 00:05:44,270 --> 00:05:46,200 of the discrete delta are the same. 100 00:05:46,200 --> 00:05:47,350 Just again. 101 00:05:47,350 --> 00:05:49,470 And what are they? 102 00:05:49,470 --> 00:05:51,800 So it picks out this column, but of course it 103 00:05:51,800 --> 00:05:58,450 divides by N, so the answer was 1/N [1, 1, 1, 1]. 104 00:05:58,450 --> 00:06:04,160 It's just constant with the 1/N, where in the continuous case we 105 00:06:04,160 --> 00:06:05,450 had 1/(2pi). 106 00:06:05,450 --> 00:06:06,880 No problem. 107 00:06:06,880 --> 00:06:08,090 OK. 108 00:06:08,090 --> 00:06:12,160 And, of course, everybody knows, suppose that I now 109 00:06:12,160 --> 00:06:15,410 start with these coefficients and add back 110 00:06:15,410 --> 00:06:16,800 to get the function. 111 00:06:16,800 --> 00:06:20,110 What would I get? 112 00:06:20,110 --> 00:06:23,900 Because, just to be sure that we believe that F and F 113 00:06:23,900 --> 00:06:26,890 inverse really are what they are supposed to be. 114 00:06:26,890 --> 00:06:29,870 If I start with these coefficients, 115 00:06:29,870 --> 00:06:39,490 add back to put those in here and reconstruct, so 1/N [ 1, 1, 116 00:06:39,490 --> 00:06:43,510 1], what will I get? 117 00:06:43,510 --> 00:06:46,960 Well, what am I supposed to get? 118 00:06:46,960 --> 00:06:47,920 The delta, right? 119 00:06:47,920 --> 00:06:50,900 I'm supposed to get back to y. 120 00:06:50,900 --> 00:06:54,900 If I started with that, did F inverse 121 00:06:54,900 --> 00:06:57,280 to get the coefficients, that was the discrete Fourier 122 00:06:57,280 --> 00:07:02,970 transform, now I add back to get, add the Fourier series up 123 00:07:02,970 --> 00:07:04,760 again to come back here, well I'll 124 00:07:04,760 --> 00:07:09,820 certainly get [1, 0, 0, 0], and you see why? 125 00:07:09,820 --> 00:07:15,540 If I multiply F, that zeroth row of F is [1, 1, 1, 1], 126 00:07:15,540 --> 00:07:19,970 times [1, 1, 1, 1] will give me N. The N will cancel. 127 00:07:19,970 --> 00:07:21,240 I get the one. 128 00:07:21,240 --> 00:07:25,260 And all the other guys add to zeroes. 129 00:07:25,260 --> 00:07:31,870 So, sure enough, it works. 130 00:07:31,870 --> 00:07:35,680 We're really just seeing an example, an important example 131 00:07:35,680 --> 00:07:37,730 of the DFT. 132 00:07:37,730 --> 00:07:40,940 And the homework, then, would have some other examples. 133 00:07:40,940 --> 00:07:44,970 But I've forgotten whether the homework has this example. 134 00:07:44,970 --> 00:07:47,370 But let's think about it now. 135 00:07:47,370 --> 00:07:54,050 Suppose that's my function value instead. 136 00:07:54,050 --> 00:07:59,210 OK, so now I'm starting with the delta. 137 00:07:59,210 --> 00:08:02,520 Again it's a delta, but it's moved over. 138 00:08:02,520 --> 00:08:04,820 And I could ask, I really should ask 139 00:08:04,820 --> 00:08:09,020 first, in the continuous case, suppose I, 140 00:08:09,020 --> 00:08:13,250 I can do it with just a little erasing here. 141 00:08:13,250 --> 00:08:15,630 Let me do the continuous case for the delta 142 00:08:15,630 --> 00:08:18,000 that we met first in this course. 143 00:08:18,000 --> 00:08:20,150 I'll shift it to a. 144 00:08:20,150 --> 00:08:25,650 If I shift the delta function to a point a, 145 00:08:25,650 --> 00:08:29,070 well, I said we'd met the delta function first in this course. 146 00:08:29,070 --> 00:08:31,130 At a, good. 147 00:08:31,130 --> 00:08:35,680 But when we did it wasn't 2pi periodic. 148 00:08:35,680 --> 00:08:41,020 So we still have, in fact, the Fourier integrals next week. 149 00:08:41,020 --> 00:08:42,750 Will have a similar formula. 150 00:08:42,750 --> 00:08:45,810 The integral will go from minus infinity to infinity 151 00:08:45,810 --> 00:08:49,130 and then we'll have the real delta, not periodic. 152 00:08:49,130 --> 00:08:53,320 Here, we have, and people call it a train of deltas. 153 00:08:53,320 --> 00:08:55,880 A train of spikes. 154 00:08:55,880 --> 00:08:58,180 Sort of you have one every 2pi. 155 00:08:58,180 --> 00:09:00,180 Anyway, that's what we've got. 156 00:09:00,180 --> 00:09:03,840 Now, you can see the answer. 157 00:09:03,840 --> 00:09:09,450 This is like perfect practice in doing an integral with a delta. 158 00:09:09,450 --> 00:09:15,220 What's the integral equal? 159 00:09:15,220 --> 00:09:18,830 Well, the spike is at x=a. 160 00:09:18,830 --> 00:09:25,560 So it picks out this function at x=a, which is e^(-ika). 161 00:09:25,560 --> 00:09:30,640 162 00:09:30,640 --> 00:09:32,540 So not constant any more. 163 00:09:32,540 --> 00:09:35,370 They depend on k. 164 00:09:35,370 --> 00:09:38,210 The 1/(2pi) is still there. 165 00:09:38,210 --> 00:09:43,270 So it's-- But still, the delta function shifted over. 166 00:09:43,270 --> 00:09:45,060 I mean, it didn't change energy. 167 00:09:45,060 --> 00:09:49,820 It didn't change, it just changed phase, so to speak. 168 00:09:49,820 --> 00:10:00,780 And we see that-- I would call this like a modulation. 169 00:10:00,780 --> 00:10:04,250 So it's staying of absolute value one, still. 170 00:10:04,250 --> 00:10:07,790 But it's not the number one, it's going around the circle. 171 00:10:07,790 --> 00:10:10,480 Going around the unit circle. 172 00:10:10,480 --> 00:10:13,700 So it's a phase factor, right. 173 00:10:13,700 --> 00:10:15,970 And that's what I'm going to expect to see here 174 00:10:15,970 --> 00:10:17,700 in the discrete case too. 175 00:10:17,700 --> 00:10:21,690 If I do this multiplication by one there, 176 00:10:21,690 --> 00:10:24,610 it picks out this column, right? 177 00:10:24,610 --> 00:10:26,840 That one will pick out this column, 178 00:10:26,840 --> 00:10:31,600 so you see it's-- Maybe I come up here now. 179 00:10:31,600 --> 00:10:39,500 Shall I just-- When I pick out that column, 180 00:10:39,500 --> 00:10:44,870 the answer then, I guess I've got the column circle, 181 00:10:44,870 --> 00:10:45,580 there it is. 182 00:10:45,580 --> 00:10:49,740 Minus i, minus i squared, minus i cubed. 183 00:10:49,740 --> 00:10:53,430 You see it's like k equals zero, one, two, three. 184 00:10:53,430 --> 00:10:56,330 Just the way here, we had k, well we 185 00:10:56,330 --> 00:11:01,570 had all integers k in that function case. 186 00:11:01,570 --> 00:11:04,560 Here we've got four integers, k equals zero, one, 187 00:11:04,560 --> 00:11:10,250 two, and three, but again it's the minus i, it's the e to the, 188 00:11:10,250 --> 00:11:13,250 it's the w bar. 189 00:11:13,250 --> 00:11:18,880 In other words, the answer was one, w bar, w bar squared, 190 00:11:18,880 --> 00:11:19,920 w bar cubed. 191 00:11:19,920 --> 00:11:24,220 Just the powers of w with this factor 192 00:11:24,220 --> 00:11:28,280 1/N. Here we had a modulation. 193 00:11:28,280 --> 00:11:31,590 It's the same picture. 194 00:11:31,590 --> 00:11:33,060 Absolute value's one. 195 00:11:33,060 --> 00:11:34,520 And and what about energy? 196 00:11:34,520 --> 00:11:42,070 Having mentioned energy, so that's another key rule. 197 00:11:42,070 --> 00:11:47,730 The key rules for the Fourier series, just let's think back. 198 00:11:47,730 --> 00:11:49,720 What were the key rules? 199 00:11:49,720 --> 00:11:52,020 First, the rule to find the coefficients. 200 00:11:52,020 --> 00:11:53,040 Good. 201 00:11:53,040 --> 00:11:56,010 Then the rule for the derivatives. 202 00:11:56,010 --> 00:11:59,970 This is so important. 203 00:11:59,970 --> 00:12:03,070 These are rules. 204 00:12:03,070 --> 00:12:05,510 Let's say, for Fourier series. 205 00:12:05,510 --> 00:12:08,560 For Fourier series. 206 00:12:08,560 --> 00:12:11,850 Let's just make this a quick review. 207 00:12:11,850 --> 00:12:14,380 What were the important rules? 208 00:12:14,380 --> 00:12:18,590 The important rules were, if I had the Fourier series of f. 209 00:12:18,590 --> 00:12:20,680 Start with the Fourier series of f. 210 00:12:20,680 --> 00:12:24,490 Then the question was, what's the Fourier series of df/dx. 211 00:12:24,490 --> 00:12:28,070 212 00:12:28,070 --> 00:12:31,580 And now I'm saying the next important rule 213 00:12:31,580 --> 00:12:38,170 is the Fourier series of f shifted. 214 00:12:38,170 --> 00:12:45,530 And then the last important rule is the energy. 215 00:12:45,530 --> 00:12:49,340 OK, and let's just, maybe this is a bad idea 216 00:12:49,340 --> 00:12:55,510 to, since we're kind of doing all of Fourier in, 217 00:12:55,510 --> 00:12:57,700 it's coming in three parts. 218 00:12:57,700 --> 00:13:05,840 Functions, discrete, integrals, but they all match. 219 00:13:05,840 --> 00:13:10,560 So this is what happens to the function. 220 00:13:10,560 --> 00:13:12,260 What happens to the coefficient? 221 00:13:12,260 --> 00:13:16,130 So this starts with coefficient c_k, 222 00:13:16,130 --> 00:13:20,410 for f, what are the coefficients for the derivative, 223 00:13:20,410 --> 00:13:22,580 just remind me? 224 00:13:22,580 --> 00:13:32,480 If f(x), so I'm starting with f(x) equals sum of c_k*e^(ikx). 225 00:13:32,480 --> 00:13:33,660 Start with that. 226 00:13:33,660 --> 00:13:35,740 And now take the derivative. 227 00:13:35,740 --> 00:13:39,000 When I take the derivative, down comes ik. 228 00:13:39,000 --> 00:13:41,370 So you remember that rule. 229 00:13:41,370 --> 00:13:44,050 Those are the Fourier coefficients of the derivative. 230 00:13:44,050 --> 00:13:47,150 Now what's the Fourier coefficients of the shift? 231 00:13:47,150 --> 00:13:50,790 If I've just shifted, translated the function, 232 00:13:50,790 --> 00:13:56,880 if my original x was this, now let me look at f(x-a). 233 00:13:56,880 --> 00:13:59,090 You'll see it. 234 00:13:59,090 --> 00:14:03,690 It'll jump out at us, it'll be a sum of the same 235 00:14:03,690 --> 00:14:06,240 c_k's e^(ik(x-a)). 236 00:14:06,240 --> 00:14:09,450 237 00:14:09,450 --> 00:14:12,650 So what are the Fourier coefficients of that? 238 00:14:12,650 --> 00:14:15,260 Well there is the e^(ikx). 239 00:14:15,260 --> 00:14:17,870 Whatever's multiplying it has got to be the Fourier 240 00:14:17,870 --> 00:14:23,280 coefficient, and we see it is c_k times e^(ik(-a)). 241 00:14:23,280 --> 00:14:30,230 e^(-ika) times c_k. 242 00:14:30,230 --> 00:14:32,470 Right? 243 00:14:32,470 --> 00:14:35,540 And, of course, that's just what we discovered here. 244 00:14:35,540 --> 00:14:38,910 That's just what we found there, that when we shifted the delta, 245 00:14:38,910 --> 00:14:42,050 we've multiplied by this modulation, 246 00:14:42,050 --> 00:14:46,590 this phase factor came into the Fourier coefficients. 247 00:14:46,590 --> 00:14:49,030 And now finally, the energy stuff. 248 00:14:49,030 --> 00:14:52,210 You remember the energy was, what's the energy? 249 00:14:52,210 --> 00:14:55,840 The integral from minus pi to pi, of f(x) 250 00:14:55,840 --> 00:15:02,990 squared dx is the same as the sum from minus infinity 251 00:15:02,990 --> 00:15:06,540 to infinity of the coefficient squared. 252 00:15:06,540 --> 00:15:09,410 And somebody correctly sent me an email 253 00:15:09,410 --> 00:15:13,140 to say energy and length squared are you really, 254 00:15:13,140 --> 00:15:14,320 is there much difference? 255 00:15:14,320 --> 00:15:15,240 No. 256 00:15:15,240 --> 00:15:16,140 No. 257 00:15:16,140 --> 00:15:17,840 You could say length squared here, 258 00:15:17,840 --> 00:15:19,910 I'm just using the word energy. 259 00:15:19,910 --> 00:15:23,030 Now, I left a space because I know that there's 260 00:15:23,030 --> 00:15:26,700 a stupid 2pi somewhere. 261 00:15:26,700 --> 00:15:29,690 Where does it come? 262 00:15:29,690 --> 00:15:32,600 You remember how to get this? 263 00:15:32,600 --> 00:15:35,920 You put that whole series in there, 264 00:15:35,920 --> 00:15:40,520 multiply by its complex conjugate to get squared. 265 00:15:40,520 --> 00:15:42,180 And integrate. 266 00:15:42,180 --> 00:15:42,770 Right? 267 00:15:42,770 --> 00:15:43,830 That was the idea. 268 00:15:43,830 --> 00:15:46,710 Isn't that how we figured out, we got to this? 269 00:15:46,710 --> 00:15:50,230 We started with this, length squared. 270 00:15:50,230 --> 00:15:53,280 We plugged in the Fourier series. 271 00:15:53,280 --> 00:15:57,770 This is f times f bar, so that's this times its conjugate. 272 00:15:57,770 --> 00:16:01,860 And we integrated, and all the cross terms vanished. 273 00:16:01,860 --> 00:16:08,220 And only the ones where e^(ikx) multiplied e^(-ikx) came. 274 00:16:08,220 --> 00:16:10,440 And those had c_k squared. 275 00:16:10,440 --> 00:16:14,270 And when we integrated that one, we probably got a 2pi. 276 00:16:14,270 --> 00:16:17,670 277 00:16:17,670 --> 00:16:20,710 So that's the energy inequality, right. 278 00:16:20,710 --> 00:16:21,960 For functions. 279 00:16:21,960 --> 00:16:24,520 And now what I was going to say is, 280 00:16:24,520 --> 00:16:31,090 you shouldn't miss the fact that in the discrete case, 281 00:16:31,090 --> 00:16:33,810 there'll be a similar energy inequality. 282 00:16:33,810 --> 00:16:40,150 So we had y was the Fourier matrix times c. 283 00:16:40,150 --> 00:16:44,830 Now, if I take the length squared of both, 284 00:16:44,830 --> 00:16:49,040 y, so I'm going to-- Right? 285 00:16:49,040 --> 00:16:52,990 That's the same as that. 286 00:16:52,990 --> 00:16:56,960 Now I'm going to do the same as this. 287 00:16:56,960 --> 00:16:58,790 I'm going to find the length squared, 288 00:16:58,790 --> 00:17:02,180 which will be y transpose y. 289 00:17:02,180 --> 00:17:04,390 No, it won't be y transpose y. 290 00:17:04,390 --> 00:17:09,690 What is length squared? y bar transpose y. 291 00:17:09,690 --> 00:17:11,480 I have to do that right. 292 00:17:11,480 --> 00:17:16,380 That will be, substituting, that's c bar F 293 00:17:16,380 --> 00:17:22,470 bar transpose times y is Fc. 294 00:17:22,470 --> 00:17:26,670 Just plugged it in, and now what do I use? 295 00:17:26,670 --> 00:17:31,560 The key fact, the fact that the columns are orthogonal. 296 00:17:31,560 --> 00:17:34,980 That's what made all these integrals simple, right? 297 00:17:34,980 --> 00:17:37,630 When I put that into there, a whole lot of integrals 298 00:17:37,630 --> 00:17:39,040 had to be zero. 299 00:17:39,040 --> 00:17:42,110 When I put this in, a whole lot of dot products 300 00:17:42,110 --> 00:17:42,980 have to be zero. 301 00:17:42,980 --> 00:17:47,470 Rows of F bar times columns of F, all zero, 302 00:17:47,470 --> 00:17:50,920 except when I'm hitting the same row. 303 00:17:50,920 --> 00:17:54,160 And when I'm hitting that same row I get an N. So I get, 304 00:17:54,160 --> 00:17:59,440 this is N c bar transpose c. 305 00:17:59,440 --> 00:18:04,760 And that's c squared. 306 00:18:04,760 --> 00:18:08,010 That's the energy inequality, it's just orthogonality 307 00:18:08,010 --> 00:18:10,330 once again. 308 00:18:10,330 --> 00:18:15,950 Everything in these weeks is coming out of orthogonality. 309 00:18:15,950 --> 00:18:20,860 Orthogonality is the fact that this is N times the identity. 310 00:18:20,860 --> 00:18:23,690 Right? 311 00:18:23,690 --> 00:18:30,470 Well, OK that's a quick recall of a bunch 312 00:18:30,470 --> 00:18:32,790 of stuff for functions. 313 00:18:32,790 --> 00:18:36,690 And just seeing maybe for the first time 314 00:18:36,690 --> 00:18:38,750 the discrete analogs. 315 00:18:38,750 --> 00:18:43,000 I guess I don't have a brilliant idea for the discrete analog 316 00:18:43,000 --> 00:18:45,100 of the derivative. 317 00:18:45,100 --> 00:18:47,940 Well, guess there's a natural idea, 318 00:18:47,940 --> 00:18:50,760 it would be a finite difference, but somehow that 319 00:18:50,760 --> 00:18:56,730 isn't a rule that gets like, high marks. 320 00:18:56,730 --> 00:19:01,080 But we saw the discrete analog of the shift 321 00:19:01,080 --> 00:19:03,640 and now we see the energy inequality 322 00:19:03,640 --> 00:19:08,630 is just that the length of the function squared 323 00:19:08,630 --> 00:19:13,900 is equal to N times the length of the coefficients squared. 324 00:19:13,900 --> 00:19:15,760 OK with that? 325 00:19:15,760 --> 00:19:18,800 Lots of formulas here. 326 00:19:18,800 --> 00:19:24,080 Let's see. 327 00:19:24,080 --> 00:19:25,310 Do some examples. 328 00:19:25,310 --> 00:19:27,930 I mean, these were simple examples. 329 00:19:27,930 --> 00:19:30,790 And I think the homework gives you some more. 330 00:19:30,790 --> 00:19:36,220 You should be able to take the Fourier transform and go 331 00:19:36,220 --> 00:19:37,960 backwards. 332 00:19:37,960 --> 00:19:41,960 And when we do convolution in a few minutes, 333 00:19:41,960 --> 00:19:45,100 we're certainly going to be taking the Fourier, 334 00:19:45,100 --> 00:19:47,680 we're going to be going both ways. 335 00:19:47,680 --> 00:19:51,070 And use all these facts. 336 00:19:51,070 --> 00:19:52,390 OK, I'll pause a moment. 337 00:19:52,390 --> 00:19:55,290 That's topic one. 338 00:19:55,290 --> 00:19:57,560 Topic two, fast Fourier transform. 339 00:19:57,560 --> 00:20:00,550 Wow. 340 00:20:00,550 --> 00:20:07,240 What's the good way to-- I mean, any decent machine 341 00:20:07,240 --> 00:20:10,810 comes with the FFT hardwired in. 342 00:20:10,810 --> 00:20:13,320 So you might say, OK I'll just use it. 343 00:20:13,320 --> 00:20:16,830 And that seems totally reasonable to me. 344 00:20:16,830 --> 00:20:24,790 But you might like to see just on one board, what's 345 00:20:24,790 --> 00:20:26,400 the key idea? 346 00:20:26,400 --> 00:20:30,560 What's the little bit of algebra that makes it work? 347 00:20:30,560 --> 00:20:35,170 So I'll just have one board here for the FFT 348 00:20:35,170 --> 00:20:38,140 and a little bit of algebra. 349 00:20:38,140 --> 00:20:45,290 Simple, simple but once it hit the world, 350 00:20:45,290 --> 00:20:49,660 well, computer scientists just love the recursion 351 00:20:49,660 --> 00:20:50,520 that comes in there. 352 00:20:50,520 --> 00:20:58,180 So they look for that in every possible other algorithm now. 353 00:20:58,180 --> 00:21:01,700 OK, let me see that point. 354 00:21:01,700 --> 00:21:03,470 So here's the main point. 355 00:21:03,470 --> 00:21:12,790 That if I want to take the-- multiply by F of size 1,024, 356 00:21:12,790 --> 00:21:19,140 the fast Fourier transform connects that full matrix 357 00:21:19,140 --> 00:21:21,110 to a half-full matrix. 358 00:21:21,110 --> 00:21:24,090 It connects that to the half-full matrix 359 00:21:24,090 --> 00:21:30,520 that takes the half-size transforms separately. 360 00:21:30,520 --> 00:21:36,670 So it's half-full because of these zeroes. 361 00:21:36,670 --> 00:21:37,950 That's the point. 362 00:21:37,950 --> 00:21:44,350 That the 1,024 matrix is connected to the 512 matrix. 363 00:21:44,350 --> 00:21:47,310 And what's underlying that? 364 00:21:47,310 --> 00:21:58,510 The 1,024 matrix is full of e to the 2pi*i, the w, over 1,024, 365 00:21:58,510 --> 00:21:59,010 right? 366 00:21:59,010 --> 00:22:01,820 That's the w for this guy. 367 00:22:01,820 --> 00:22:07,210 And then the w for this guy, for both of these, 368 00:22:07,210 --> 00:22:09,690 is e^(2pi*i/512). 369 00:22:09,690 --> 00:22:14,140 370 00:22:14,140 --> 00:22:16,620 So if there's a connection between that matrix 371 00:22:16,620 --> 00:22:18,180 and this matrix, there'd better be 372 00:22:18,180 --> 00:22:21,330 a connection between that number and that number. 373 00:22:21,330 --> 00:22:24,250 Because this is the number that fills this one, 374 00:22:24,250 --> 00:22:26,670 and this is the number that fills these. 375 00:22:26,670 --> 00:22:30,120 So what's the connection? 376 00:22:30,120 --> 00:22:32,530 It's just perfect, right? 377 00:22:32,530 --> 00:22:38,960 If I take this number, which is one part of the whole circle, 378 00:22:38,960 --> 00:22:43,310 1/1,024, a fraction of the whole circle, 379 00:22:43,310 --> 00:22:47,120 what do I do to get this guy? 380 00:22:47,120 --> 00:22:51,430 To get one 512th of the way around the circle? 381 00:22:51,430 --> 00:22:55,110 I square it. 382 00:22:55,110 --> 00:23:01,650 The square of this w is this w. 383 00:23:01,650 --> 00:23:07,970 Let me call this w_N, and this one w_M. 384 00:23:07,970 --> 00:23:12,930 Maybe I'll use caps, yeah I'm using cap N, this is my w. 385 00:23:12,930 --> 00:23:14,570 This is the one I want. 386 00:23:14,570 --> 00:23:19,660 The w_N, the N by N one, N is 1,024, 387 00:23:19,660 --> 00:23:23,250 and the point is, everybody saw that, when I squared it 388 00:23:23,250 --> 00:23:24,790 I doubled the angle? 389 00:23:24,790 --> 00:23:31,130 When I doubled that angle the two over that gave me 1/512. 390 00:23:31,130 --> 00:23:32,650 Fantastic. 391 00:23:32,650 --> 00:23:39,590 Of course, that doesn't make this equal to that, 392 00:23:39,590 --> 00:23:44,330 but it suggests that there is a close connection. 393 00:23:44,330 --> 00:23:48,580 So let me finish here the key idea of the Fourier 394 00:23:48,580 --> 00:23:53,580 transform in block matrix form. 395 00:23:53,580 --> 00:23:55,110 What's the key idea? 396 00:23:55,110 --> 00:23:59,430 So instead of doing the big transform, the full size, 397 00:23:59,430 --> 00:24:01,360 I'm going to do two half-sizes. 398 00:24:01,360 --> 00:24:08,520 But what am I going to apply those to? 399 00:24:08,520 --> 00:24:10,610 Here's the trick. 400 00:24:10,610 --> 00:24:16,530 These two separate guys apply to the odd-numbered coefficients. 401 00:24:16,530 --> 00:24:18,410 The odd-numbered component. 402 00:24:18,410 --> 00:24:19,710 And the even. 403 00:24:19,710 --> 00:24:24,420 So I have to first do a little permutation, 404 00:24:24,420 --> 00:24:27,950 and even comes first, always. 405 00:24:27,950 --> 00:24:32,080 Even means zero, two, four, up to a 1,022. 406 00:24:32,080 --> 00:24:38,350 So this is zero, two; zero up to 1,022. 407 00:24:38,350 --> 00:24:40,840 And then come all the odd guys. 408 00:24:40,840 --> 00:24:44,350 One up to 1,023. 409 00:24:44,350 --> 00:24:48,360 So this is a permutation, a simple permutation. 410 00:24:48,360 --> 00:24:53,060 Just take your 1,024 numbers, pick out the even ones, 411 00:24:53,060 --> 00:24:55,270 put them on top. 412 00:24:55,270 --> 00:24:56,340 Right? 413 00:24:56,340 --> 00:24:57,940 In other words, put them on top where 414 00:24:57,940 --> 00:25:00,420 512 is going to act on it. 415 00:25:00,420 --> 00:25:03,510 Put the odd ones at the bottom, the last 512, 416 00:25:03,510 --> 00:25:05,020 that guy will act on that. 417 00:25:05,020 --> 00:25:10,010 So there's 512 numbers there, with the even coefficients, 418 00:25:10,010 --> 00:25:10,750 this acts. 419 00:25:10,750 --> 00:25:14,990 OK, now we've got two half-size transforms. 420 00:25:14,990 --> 00:25:21,720 Because we're applying this to the y, to a to a typical y. 421 00:25:21,720 --> 00:25:22,630 OK. 422 00:25:22,630 --> 00:25:24,900 But I've just written F without a y 423 00:25:24,900 --> 00:25:27,230 so I don't really need a y here. 424 00:25:27,230 --> 00:25:30,130 This is a matrix identity. 425 00:25:30,130 --> 00:25:33,960 It's a matrix identity that's got a bunch of zeroes there. 426 00:25:33,960 --> 00:25:36,410 Of course, that matrix is full of zeroes. 427 00:25:36,410 --> 00:25:41,640 I mean, this is instant speed to do that permutation. 428 00:25:41,640 --> 00:25:45,040 Grab the evens, put them in front of the odds. 429 00:25:45,040 --> 00:25:47,770 OK, so now I've got two half-sizes, 430 00:25:47,770 --> 00:25:51,010 but then I have to put them back together 431 00:25:51,010 --> 00:25:53,290 to get the full-size matrix. 432 00:25:53,290 --> 00:25:55,660 And that is also a matrix. 433 00:25:55,660 --> 00:25:58,640 Turns out to be a diagonal there, 434 00:25:58,640 --> 00:26:05,770 and a minus the diagonal goes there. 435 00:26:05,770 --> 00:26:09,250 So actually, that looks great too, right? 436 00:26:09,250 --> 00:26:11,030 Full of zeroes, the identity. 437 00:26:11,030 --> 00:26:13,240 No multiplications whatever. 438 00:26:13,240 --> 00:26:16,000 Well, these are the only multiplications, 439 00:26:16,000 --> 00:26:21,490 sometimes they're called twiddle factors, give it a fancy name. 440 00:26:21,490 --> 00:26:25,330 Official sounding name, twiddle factors. 441 00:26:25,330 --> 00:26:29,690 OK, so that diagonal matrix D, what's that? 442 00:26:29,690 --> 00:26:36,010 That diagonal matrix D happens to be just the powers of w, 443 00:26:36,010 --> 00:26:41,770 sit along D. 1, w, up to w to the, 444 00:26:41,770 --> 00:26:44,900 this is w_N, we're talking, the big w, 445 00:26:44,900 --> 00:26:50,210 and it's only half-size so it goes up to M-1. 446 00:26:50,210 --> 00:26:55,000 Half, M is half of N. Everybody's got that, right? 447 00:26:55,000 --> 00:26:57,290 M is half of N here. 448 00:26:57,290 --> 00:27:01,130 I'll get that written on the board. 449 00:27:01,130 --> 00:27:06,490 M is 512, N is 1,024, here we have the powers 450 00:27:06,490 --> 00:27:10,810 of this guy up to 511. 451 00:27:10,810 --> 00:27:17,670 The total size being 512 because that's a 512 by 512 matrix. 452 00:27:17,670 --> 00:27:21,290 I guess I can remember somewhere, 453 00:27:21,290 --> 00:27:23,780 being at a conference, this was probably soon 454 00:27:23,780 --> 00:27:29,780 after the FFT became sort of famous. 455 00:27:29,780 --> 00:27:33,750 And then somebody who was just presenting the idea 456 00:27:33,750 --> 00:27:38,860 and as soon as it was presented that way, I was happy. 457 00:27:38,860 --> 00:27:39,760 I guess. 458 00:27:39,760 --> 00:27:43,020 I thought OK, there you see the idea. 459 00:27:43,020 --> 00:27:46,770 Permutation, reorder the even-odd. 460 00:27:46,770 --> 00:27:50,220 Two half-size transforms, put them back together. 461 00:27:50,220 --> 00:27:52,780 And what's happened here? 462 00:27:52,780 --> 00:27:58,760 The work of this matrix, multiplying 463 00:27:58,760 --> 00:28:01,520 by this matrix, which would be 1,024 squared 464 00:28:01,520 --> 00:28:04,560 is now practically cut in half. 465 00:28:04,560 --> 00:28:06,290 Because this is nothing. 466 00:28:06,290 --> 00:28:10,310 And we have just this diagonal multiplication to do, 467 00:28:10,310 --> 00:28:12,900 and of course this is the same as that, just with minus signs. 468 00:28:12,900 --> 00:28:15,890 So the total multiplications we have to do, 469 00:28:15,890 --> 00:28:19,750 the total number of twiddle factors, is just 512, 470 00:28:19,750 --> 00:28:22,340 and then we're golden. 471 00:28:22,340 --> 00:28:27,690 So we've got half the work plus 512, 512 operations. 472 00:28:27,690 --> 00:28:30,540 That's pretty good. 473 00:28:30,540 --> 00:28:33,060 And of course it gets better. 474 00:28:33,060 --> 00:28:35,980 How? 475 00:28:35,980 --> 00:28:38,552 Once you have the idea of getting down to 512, 476 00:28:38,552 --> 00:28:39,760 what are you going to do now? 477 00:28:39,760 --> 00:28:46,070 This is the computer scientist's favorite idea. 478 00:28:46,070 --> 00:28:47,540 Do it again. 479 00:28:47,540 --> 00:28:48,900 That's what it comes to. 480 00:28:48,900 --> 00:28:53,200 Whatever worked for 1,024 to get to 512 is going to work. 481 00:28:53,200 --> 00:28:58,720 So now I'll split this up into, well, each 512, 482 00:28:58,720 --> 00:29:01,350 so now I have to do, yeah. 483 00:29:01,350 --> 00:29:07,860 Let me write F_512 will be, it's now smaller. 484 00:29:07,860 --> 00:29:15,370 An I, an I, and a D and a minus D for the 512 size, 485 00:29:15,370 --> 00:29:22,480 of F_256, 256 and then the permutation, 486 00:29:22,480 --> 00:29:26,480 the odd-even permutation P. So we're 487 00:29:26,480 --> 00:29:30,960 doing this idea in there, and in there. 488 00:29:30,960 --> 00:29:38,700 So it's just recursive. 489 00:29:38,700 --> 00:29:39,200 Recursive. 490 00:29:39,200 --> 00:29:41,920 And now, if we go all the way, so you 491 00:29:41,920 --> 00:29:46,400 see why I keep taking powers of two. 492 00:29:46,400 --> 00:29:48,390 It's natural to have powers of two. 493 00:29:48,390 --> 00:29:49,410 Two or three. 494 00:29:49,410 --> 00:29:51,190 Three is also good. 495 00:29:51,190 --> 00:29:53,560 I mean, all this gets so optimized 496 00:29:53,560 --> 00:29:57,880 that powers of two or three are pretty good. 497 00:29:57,880 --> 00:30:00,320 And you just use the same idea. 498 00:30:00,320 --> 00:30:06,720 There'd be a similar idea here for, if I was doing instead 499 00:30:06,720 --> 00:30:12,110 of odd-even, even-odd I was doing maybe three groups. 500 00:30:12,110 --> 00:30:19,600 But stick with two, that's fine. 501 00:30:19,600 --> 00:30:24,130 Then you might ask, if you were a worrier 502 00:30:24,130 --> 00:30:30,650 I guess you might ask what if it's not a power of two. 503 00:30:30,650 --> 00:30:32,800 I think you just add in zeroes. 504 00:30:32,800 --> 00:30:41,010 Just pad it out to be the next power of two. 505 00:30:41,010 --> 00:30:43,760 Nothing difficult there. 506 00:30:43,760 --> 00:30:44,920 I think that's right. 507 00:30:44,920 --> 00:30:49,750 Hope that's right. 508 00:30:49,750 --> 00:30:55,690 And once this idea came out, of course, people started looking. 509 00:30:55,690 --> 00:31:00,110 What if the number here was prime? 510 00:31:00,110 --> 00:31:05,790 And found another neat bit of algebra 511 00:31:05,790 --> 00:31:08,020 that worked OK for prime numbers. 512 00:31:08,020 --> 00:31:11,680 Using a little bit of number theory. 513 00:31:11,680 --> 00:31:16,290 But the ultimate winner was this one. 514 00:31:16,290 --> 00:31:23,680 So maybe I'll just refer you to those pages 515 00:31:23,680 --> 00:31:29,700 in the book, where you'll spot this matrix equality. 516 00:31:29,700 --> 00:31:33,230 And then next to it is the algebra 517 00:31:33,230 --> 00:31:37,200 that you have to do to check it. 518 00:31:37,200 --> 00:31:42,910 I can just say, because I want you to look to the right spot 519 00:31:42,910 --> 00:31:47,190 there, maybe I'll take out the great-- This is sometimes 520 00:31:47,190 --> 00:31:53,200 called after Parseval, or some other person. 521 00:31:53,200 --> 00:31:56,850 Yeah, the algebra-- 522 00:31:56,850 --> 00:32:02,820 Let me start the algebra that made this thing work. 523 00:32:02,820 --> 00:32:07,800 We want the sum, when we multiply Fourier, 524 00:32:07,800 --> 00:32:14,760 F times something, we want the sum of w^(jk), right? 525 00:32:14,760 --> 00:32:23,040 That's the coefficient, that's the entry of F. Times c_k. 526 00:32:23,040 --> 00:32:26,930 Summed from k=0 to N-1. 527 00:32:26,930 --> 00:32:30,990 To 1,023. 528 00:32:30,990 --> 00:32:40,220 That's the y_j that we're trying to compute. 529 00:32:40,220 --> 00:32:45,520 We're computing 1,024 y's from 1,024 c's by adding up 530 00:32:45,520 --> 00:32:49,130 the Fourier series when we multiply by F. This is F, 531 00:32:49,130 --> 00:33:00,770 this is the equation y=Fc written with subscripts. 532 00:33:00,770 --> 00:33:02,630 So this is what the matrices are doing, 533 00:33:02,630 --> 00:33:07,150 and now where do I find that 512 thing? 534 00:33:07,150 --> 00:33:10,050 How do I get M into the picture, remembering 535 00:33:10,050 --> 00:33:16,910 that the w_N squared was w_M, right? 536 00:33:16,910 --> 00:33:20,060 That's what we saw. 537 00:33:20,060 --> 00:33:23,140 This is the big number, this is half of it, 538 00:33:23,140 --> 00:33:26,750 so this is a little bit of the part around the circle. 539 00:33:26,750 --> 00:33:29,590 When I go twice as far I get to the other one. 540 00:33:29,590 --> 00:33:34,510 So that's the thing that we've got to use. 541 00:33:34,510 --> 00:33:37,430 Everything is going to depend on that. 542 00:33:37,430 --> 00:33:41,230 So this was w_N, of course. 543 00:33:41,230 --> 00:33:43,750 This is the N by N transform. 544 00:33:43,750 --> 00:33:47,010 So now comes what, what's the key idea? 545 00:33:47,010 --> 00:33:49,960 The key idea is split into even and odd. 546 00:33:49,960 --> 00:33:51,640 And then use this. 547 00:33:51,640 --> 00:33:55,160 So split into the even ones and the odd ones. 548 00:33:55,160 --> 00:34:01,060 So I write this as two separate sums, a sum for the even ones. 549 00:34:01,060 --> 00:34:10,520 w_N, now, so now the even c_k, so I'm going to multiply 550 00:34:10,520 --> 00:34:14,330 by c_(2k). 551 00:34:14,330 --> 00:34:16,820 These are the ones with even, and the sum 552 00:34:16,820 --> 00:34:23,070 is only going to go from zero to M-1, right? 553 00:34:23,070 --> 00:34:25,080 This is only half of the terms. 554 00:34:25,080 --> 00:34:31,090 And then look on the other half, plus the same sum 555 00:34:31,090 --> 00:34:32,340 of-- But I didn't finish here. 556 00:34:32,340 --> 00:34:33,280 Let me finish. 557 00:34:33,280 --> 00:34:35,870 So I'm just picking out, I'm taking, 558 00:34:35,870 --> 00:34:38,320 instead of k I'm doing 2k. 559 00:34:38,320 --> 00:34:43,430 So I have j times 2k here. 560 00:34:43,430 --> 00:34:49,460 And now these will be the odd ones. c_(2k+1), 561 00:34:49,460 --> 00:34:54,990 and omega_N to the j(2k+1). 562 00:34:54,990 --> 00:35:00,670 563 00:35:00,670 --> 00:35:03,330 It's a lot to ask you. 564 00:35:03,330 --> 00:35:11,530 To focus on this bit of algebra. 565 00:35:11,530 --> 00:35:14,230 I hope you're going to go away feeling, 566 00:35:14,230 --> 00:35:17,170 well it's pretty darn simple. 567 00:35:17,170 --> 00:35:18,970 I mean there'll be a lot of indices, 568 00:35:18,970 --> 00:35:22,650 and if I push it all the way through there'll be a few more. 569 00:35:22,650 --> 00:35:25,250 But the point is, it's pretty darn simple. 570 00:35:25,250 --> 00:35:26,730 For example, this term. 571 00:35:26,730 --> 00:35:29,700 What have I got there? 572 00:35:29,700 --> 00:35:33,450 Look at that term, that's beautiful. 573 00:35:33,450 --> 00:35:36,620 That's w_N squared. 574 00:35:36,620 --> 00:35:40,650 What is w_N squared? 575 00:35:40,650 --> 00:35:42,630 It's w_M. 576 00:35:42,630 --> 00:35:45,060 So instead of w_N squared, I'm going 577 00:35:45,060 --> 00:35:49,680 to replace that w_N squared by w_M. 578 00:35:49,680 --> 00:35:52,160 And the sum goes from zero to M-1, 579 00:35:52,160 --> 00:35:56,050 and what does that represent? 580 00:35:56,050 --> 00:35:58,980 That represents the F_512 multiplication, 581 00:35:58,980 --> 00:36:00,840 the half-size transform. 582 00:36:00,840 --> 00:36:04,280 It goes halfway, it operates on the even ones, 583 00:36:04,280 --> 00:36:09,740 and it uses the M. It's just perfectly, 584 00:36:09,740 --> 00:36:13,980 so this is nothing but the Fourier matrix, 585 00:36:13,980 --> 00:36:21,950 the M by M Fourier matrix, acting on the even c's. 586 00:36:21,950 --> 00:36:23,110 That's what that is. 587 00:36:23,110 --> 00:36:26,630 And that's why we get these identities. 588 00:36:26,630 --> 00:36:28,310 That's why we get these identities, 589 00:36:28,310 --> 00:36:33,650 because they're acting on the even-- The top half is 590 00:36:33,650 --> 00:36:37,470 the even guy. 591 00:36:37,470 --> 00:36:40,110 OK, this is almost as good. 592 00:36:40,110 --> 00:36:42,410 This is the odd c's. 593 00:36:42,410 --> 00:36:47,900 Here I have, now do I have w_M here? 594 00:36:47,900 --> 00:36:49,530 I've got to have w_M. 595 00:36:49,530 --> 00:36:52,730 596 00:36:52,730 --> 00:36:54,210 Well, you can see. 597 00:36:54,210 --> 00:36:58,300 It's not quite coming out right, and that's the twiddle factor. 598 00:36:58,300 --> 00:37:05,200 I have to take out a w_N to the power j to make things good. 599 00:37:05,200 --> 00:37:08,470 And when I take out that w_N to the power j, 600 00:37:08,470 --> 00:37:14,820 that's what goes in the D, in the diagonal matrix. 601 00:37:14,820 --> 00:37:16,010 Yeah. 602 00:37:16,010 --> 00:37:18,610 I won't go more than that. 603 00:37:18,610 --> 00:37:24,380 You couldn't, there's no reason to do everything here 604 00:37:24,380 --> 00:37:27,620 on the board when the main point is there. 605 00:37:27,620 --> 00:37:30,250 And then the point was recursion and then, oh, 606 00:37:30,250 --> 00:37:33,180 let me complete the recursion. 607 00:37:33,180 --> 00:37:38,520 So I recurse down to 256, then 128, then 64. 608 00:37:38,520 --> 00:37:42,390 And what do I get in the end? 609 00:37:42,390 --> 00:37:45,990 What do I get altogether, once I've got all the way down 610 00:37:45,990 --> 00:37:48,440 to size two? 611 00:37:48,440 --> 00:37:54,830 I have a whole lot of these factors 612 00:37:54,830 --> 00:37:58,400 down in the middle, the part that 613 00:37:58,400 --> 00:38:03,060 used to be hard is now down to F_2 or F_1 or something. 614 00:38:03,060 --> 00:38:07,340 So let's say F_1, one by one, just the identity. 615 00:38:07,340 --> 00:38:09,050 So I go all the way. 616 00:38:09,050 --> 00:38:14,000 Ten steps from 1024, 512, 256, every time 617 00:38:14,000 --> 00:38:16,090 I get twiddle factors. 618 00:38:16,090 --> 00:38:19,200 Every time I get P's. 619 00:38:19,200 --> 00:38:25,150 A lot of P's, but the F_512, what used to be the hard part, 620 00:38:25,150 --> 00:38:29,170 has gone to the easy part. 621 00:38:29,170 --> 00:38:31,250 And then what do I have? 622 00:38:31,250 --> 00:38:33,910 I've got just a permutation there. 623 00:38:33,910 --> 00:38:38,920 And this is the actual work. 624 00:38:38,920 --> 00:38:43,490 This is the only work left, is the matrices 625 00:38:43,490 --> 00:38:49,180 like this, for different sizes. 626 00:38:49,180 --> 00:38:50,886 And I have to do those. 627 00:38:50,886 --> 00:38:52,510 I have to do all those twiddle factors. 628 00:38:52,510 --> 00:38:58,300 So how many matrices are there, there? 629 00:38:58,300 --> 00:39:00,310 It's the log, right? 630 00:39:00,310 --> 00:39:02,480 Every time I divided by two. 631 00:39:02,480 --> 00:39:06,480 So if I started at 1,024, I do ten times, 632 00:39:06,480 --> 00:39:11,580 I have ten of those matrices and have me down to N=1. 633 00:39:11,580 --> 00:39:18,350 So I've got ten of these, and each one takes 1,024 or maybe 634 00:39:18,350 --> 00:39:19,390 only half of that. 635 00:39:19,390 --> 00:39:22,790 Actually, only half because this is a copy of that. 636 00:39:22,790 --> 00:39:26,660 I think the final count, and can I just put it here, 637 00:39:26,660 --> 00:39:32,350 this is the great number, is each of these took 638 00:39:32,350 --> 00:39:35,400 N multiplications. 639 00:39:35,400 --> 00:39:38,510 But there were only log to the base two-- Oh, no. 640 00:39:38,510 --> 00:39:40,990 Each of them took half of N multiplications, 641 00:39:40,990 --> 00:39:44,680 because the D and the minus D are just, 642 00:39:44,680 --> 00:39:46,240 I don't have to repeat. 643 00:39:46,240 --> 00:39:50,320 So half N for each factor and the number of factors 644 00:39:50,320 --> 00:39:55,360 is log to the base two of N. Ten, for 1,024. 645 00:39:55,360 --> 00:40:00,120 So that's the magic of the FFT. 646 00:40:00,120 --> 00:40:02,980 OK. 647 00:40:02,980 --> 00:40:06,920 It's almost all on one board, one and a half boards. 648 00:40:06,920 --> 00:40:21,000 To tell you the key point, odds and evens, recursion, 649 00:40:21,000 --> 00:40:25,270 twiddle factors, getting down to the point where you only 650 00:40:25,270 --> 00:40:30,100 have twiddle factors left and then those multiplications 651 00:40:30,100 --> 00:40:37,640 are only N log N. Good? 652 00:40:37,640 --> 00:40:38,230 Yes. 653 00:40:38,230 --> 00:40:40,180 Right, OK. 654 00:40:40,180 --> 00:40:47,840 Now, that's discrete transform. 655 00:40:47,840 --> 00:40:52,870 The theory behind it and the fantastic algorithm 656 00:40:52,870 --> 00:40:59,000 that executes it. 657 00:40:59,000 --> 00:41:00,600 Are you ready for a convolution? 658 00:41:00,600 --> 00:41:04,570 Can we start on a topic that's really quite nice, 659 00:41:04,570 --> 00:41:10,130 and then Friday will be the focus on convolution. 660 00:41:10,130 --> 00:41:14,210 Friday will certainly be all convolution day. 661 00:41:14,210 --> 00:41:21,920 But maybe it's not a bad idea to see now, what's the question? 662 00:41:21,920 --> 00:41:25,310 Let me ask that question. 663 00:41:25,310 --> 00:41:27,990 OK, convolution. 664 00:41:27,990 --> 00:41:35,200 So we're into the next section of the book, Section 4.4, 665 00:41:35,200 --> 00:41:42,050 it must be. 666 00:41:42,050 --> 00:41:46,510 And let me do it first for a Fourier series. 667 00:41:46,510 --> 00:41:50,690 I have convolution of series, convolution of discrete. 668 00:41:50,690 --> 00:41:57,490 Convolution of integrals, but we haven't got there yet. 669 00:41:57,490 --> 00:42:02,370 So I'll do this one, this series. 670 00:42:02,370 --> 00:42:05,090 So let me start with a couple of series. 671 00:42:05,090 --> 00:42:08,430 f(x) is the sum of c_k*e^(ikx), say. 672 00:42:08,430 --> 00:42:11,070 673 00:42:11,070 --> 00:42:18,430 g(x) is the sum of some other coefficients. 674 00:42:18,430 --> 00:42:21,600 And I'm going to ask you a simple question. 675 00:42:21,600 --> 00:42:27,550 What are the Fourier coefficients of f times g? 676 00:42:27,550 --> 00:42:39,990 If I multiply those functions, equals something. 677 00:42:39,990 --> 00:42:44,490 And let me call those coefficients, h maybe. 678 00:42:44,490 --> 00:42:54,580 h_k*e^(ikx), and my question is what are the coefficients h_k 679 00:42:54,580 --> 00:42:59,590 of f times g? 680 00:42:59,590 --> 00:43:10,550 That's the question that convolution answers. 681 00:43:10,550 --> 00:43:14,870 Actually, both this series and the discrete series 682 00:43:14,870 --> 00:43:17,290 are highly interesting. 683 00:43:17,290 --> 00:43:20,770 Highly interesting. 684 00:43:20,770 --> 00:43:25,740 So here I wrote it for this series. 685 00:43:25,740 --> 00:43:34,540 If I write it for the discrete ones, you'll see, so let 686 00:43:34,540 --> 00:43:36,310 me do it over here for the discrete one. 687 00:43:36,310 --> 00:43:39,850 Because I can write it out for the discrete ones. 688 00:43:39,850 --> 00:43:47,530 My y's are c_0 plus c_1 e t-- No, w. 689 00:43:47,530 --> 00:43:58,870 I have this nice notation, w. plus c_(N-1)*w^(N-1), right? 690 00:43:58,870 --> 00:44:03,440 That's the-- Ooh. 691 00:44:03,440 --> 00:44:05,630 What's that? 692 00:44:05,630 --> 00:44:08,000 I haven't got that right. 693 00:44:08,000 --> 00:44:12,540 Yes, what do I want now? 694 00:44:12,540 --> 00:44:15,660 I need, yep. 695 00:44:15,660 --> 00:44:18,800 Sorry, I'm looking, really I'm looking at y_j, 696 00:44:18,800 --> 00:44:20,960 the j-th component of y. 697 00:44:20,960 --> 00:44:32,510 So I need a w^j, w^(j(N-1).) Yeah, OK, let me-- OK. 698 00:44:32,510 --> 00:44:33,010 Alright. 699 00:44:33,010 --> 00:44:36,420 And so that's my f. 700 00:44:36,420 --> 00:44:44,830 I'll come back to that, let me stay with this. 701 00:44:44,830 --> 00:44:47,530 I'll stay with this to make the main point, 702 00:44:47,530 --> 00:44:51,230 and then Friday we'll see it in a neat way 703 00:44:51,230 --> 00:44:53,960 for the discrete one. 704 00:44:53,960 --> 00:44:56,300 So I'm coming back to this. f has 705 00:44:56,300 --> 00:44:59,940 its Fourier series. g has its Fourier series. 706 00:44:59,940 --> 00:45:02,870 I multiply. 707 00:45:02,870 --> 00:45:10,080 What happens when I multiply this times this? 708 00:45:10,080 --> 00:45:12,430 I'm not going to integrate. 709 00:45:12,430 --> 00:45:14,890 I mean, when I do this multiplication, 710 00:45:14,890 --> 00:45:18,890 I'm going to get a mass of terms. 711 00:45:18,890 --> 00:45:21,230 A real lot of terms. 712 00:45:21,230 --> 00:45:23,650 And I'm not going to integrate them away. 713 00:45:23,650 --> 00:45:25,850 So they're all there. 714 00:45:25,850 --> 00:45:27,160 So what am I asking? 715 00:45:27,160 --> 00:45:31,320 I'm asking to pick out all the terms that have 716 00:45:31,320 --> 00:45:36,050 the same exponential with them. 717 00:45:36,050 --> 00:45:38,730 Like, what's h_0? 718 00:45:38,730 --> 00:45:42,980 Yes, tell me what h_0 is? 719 00:45:42,980 --> 00:45:45,200 If you can pick out h_0 here, you'll 720 00:45:45,200 --> 00:45:50,630 get the idea of convolution. 721 00:45:50,630 --> 00:45:53,460 What's the constant term if I multiply 722 00:45:53,460 --> 00:45:57,710 this mess times this mess, and I look 723 00:45:57,710 --> 00:46:04,420 for the constant term, h_0, where 724 00:46:04,420 --> 00:46:07,080 do I get the constant terms when I multiply that by that? 725 00:46:07,080 --> 00:46:08,490 Just think about that. 726 00:46:08,490 --> 00:46:11,180 Where do I get a constant, without any k, 727 00:46:11,180 --> 00:46:13,960 without an e^(ikx)? 728 00:46:13,960 --> 00:46:16,650 If I multiply that by that. 729 00:46:16,650 --> 00:46:21,970 Well tell me one place I get something. c_0 times? d_0. 730 00:46:21,970 --> 00:46:25,650 Good. 731 00:46:25,650 --> 00:46:28,030 Is that the end of the story? 732 00:46:28,030 --> 00:46:29,130 No. 733 00:46:29,130 --> 00:46:32,159 If you thought that multiplying the functions, 734 00:46:32,159 --> 00:46:33,950 I just multiplied the Fourier coefficients, 735 00:46:33,950 --> 00:46:36,820 the first point is no. 736 00:46:36,820 --> 00:46:38,420 There's more stuff. 737 00:46:38,420 --> 00:46:41,940 Where else do I get a constant out of this? 738 00:46:41,940 --> 00:46:44,850 Just look at it, do that multiplication 739 00:46:44,850 --> 00:46:48,100 and ask yourself where's the constant. 740 00:46:48,100 --> 00:46:49,090 Another one, yep. 741 00:46:49,090 --> 00:46:54,890 You were going to say it is? c_1 times d_(-1). 742 00:46:54,890 --> 00:46:55,580 Right. 743 00:46:55,580 --> 00:46:59,750 Right. c_1 times d_(-1). 744 00:46:59,750 --> 00:47:05,890 And tell me all of them, now. c_2 times d_(-2). 745 00:47:05,890 --> 00:47:09,112 And what about c_(-1)? 746 00:47:09,112 --> 00:47:09,820 There's a c_(-1). 747 00:47:09,820 --> 00:47:12,740 748 00:47:12,740 --> 00:47:16,150 It multiplies d_1. 749 00:47:16,150 --> 00:47:19,500 And onwards. 750 00:47:19,500 --> 00:47:26,330 So the coefficient comes from, now how 751 00:47:26,330 --> 00:47:29,650 could you describe that? 752 00:47:29,650 --> 00:47:35,580 I guess I'll describe it as, I'll need a sum to multiply. 753 00:47:35,580 --> 00:47:42,490 This will be the sum of c_k times d what? 754 00:47:42,490 --> 00:47:45,600 Minus k, right? 755 00:47:45,600 --> 00:47:49,370 That's what you told me, the piece at the start, 756 00:47:49,370 --> 00:47:52,170 and that's the pattern that keeps going. 757 00:47:52,170 --> 00:47:58,810 OK, that's h_0, the sum of c_k times d_(-k). 758 00:47:58,810 --> 00:48:05,580 Now, we have just time to do the next one. 759 00:48:05,580 --> 00:48:07,310 We've got time but not space, where 760 00:48:07,310 --> 00:48:09,640 the heck am I going to put it? 761 00:48:09,640 --> 00:48:14,550 I want to do h_k, I guess. 762 00:48:14,550 --> 00:48:18,170 Or I better use a different letter h_l, 763 00:48:18,170 --> 00:48:22,890 let me use the letter h_l, and God there's no space. 764 00:48:22,890 --> 00:48:31,120 Alright, so can I-- Yes. h_l. 765 00:48:31,120 --> 00:48:32,300 OK. 766 00:48:32,300 --> 00:48:36,810 So this was h_0, let me keep things sort of looking right 767 00:48:36,810 --> 00:48:37,470 for the moment. 768 00:48:37,470 --> 00:48:39,640 OK, now you're going to fix h_l. 769 00:48:39,640 --> 00:48:44,770 So what does c_0 multiply, if I'm looking for h_l, 770 00:48:44,770 --> 00:48:47,920 I'm looking for the coefficient of e^(ilx). 771 00:48:47,920 --> 00:48:52,080 So ask yourself how do I get e^(ilx) when that multiplies 772 00:48:52,080 --> 00:48:54,110 that? 773 00:48:54,110 --> 00:48:57,820 When that multiplies that, and I'm looking for an e^(ilx), 774 00:48:57,820 --> 00:49:02,430 I get one when c_0 all multiplies what? 775 00:49:02,430 --> 00:49:05,690 This is it. d_l, right. 776 00:49:05,690 --> 00:49:07,130 And what about for c_1? 777 00:49:07,130 --> 00:49:09,860 778 00:49:09,860 --> 00:49:14,190 Think of here, I have a c_1*e^(i1x). 779 00:49:14,190 --> 00:49:18,120 What does it multiply down here to get the exponential 780 00:49:18,120 --> 00:49:22,340 to be l? ilx? 781 00:49:22,340 --> 00:49:25,860 It doesn't multiply d_(-1). 782 00:49:25,860 --> 00:49:31,530 It multiplies, c_1 multiplies? d_(l-1). 783 00:49:31,530 --> 00:49:38,470 Good, good. l-1, right. l-1, and what are you noticing here? 784 00:49:38,470 --> 00:49:40,560 c minus, I'll have to fill that in. 785 00:49:40,560 --> 00:49:45,620 But you're seeing the pattern here? 786 00:49:45,620 --> 00:49:47,380 And what was the pattern here? 787 00:49:47,380 --> 00:49:51,870 Those numbers added to this number. 788 00:49:51,870 --> 00:49:54,490 And now these numbers add to l. 789 00:49:54,490 --> 00:50:00,040 Those numbers add to l, whatever it is, 790 00:50:00,040 --> 00:50:03,020 the two indices have to add to l, 791 00:50:03,020 --> 00:50:06,140 so that when I multiply the exponential they'll add 792 00:50:06,140 --> 00:50:08,000 to e^(ilx). 793 00:50:08,000 --> 00:50:11,480 They'll multiply to e^(ilx). 794 00:50:11,480 --> 00:50:14,360 So what goes there? 795 00:50:14,360 --> 00:50:16,750 It's probably l+1, right? 796 00:50:16,750 --> 00:50:22,300 So that l+1 combined with minus one gives me the l. 797 00:50:22,300 --> 00:50:26,560 If you tell me what goes there, I'm a happy person. 798 00:50:26,560 --> 00:50:27,750 Let's make it h_l. 799 00:50:27,750 --> 00:50:33,300 We're ready for the final formula for convolutions. 800 00:50:33,300 --> 00:50:39,020 Big star. 801 00:50:39,020 --> 00:50:44,200 To find h_l, the coefficient of e^(ilx), 802 00:50:44,200 --> 00:50:48,390 when you multiply that by that, you look at c_k, 803 00:50:48,390 --> 00:50:58,110 and which d is going to show up in the e^(ilx) term? l-k, 804 00:50:58,110 --> 00:50:59,700 is that what you said? 805 00:50:59,700 --> 00:51:02,870 I hope, yeah. l-k. 806 00:51:02,870 --> 00:51:04,590 Right, that's it. 807 00:51:04,590 --> 00:51:05,250 That's it. 808 00:51:05,250 --> 00:51:10,030 So we've got a lot of computation here. 809 00:51:10,030 --> 00:51:17,300 But we've got the idea of what, we've got a formula. 810 00:51:17,300 --> 00:51:20,740 And most of all we have the magic rule. 811 00:51:20,740 --> 00:51:25,480 In convolutions, convolutions are, 812 00:51:25,480 --> 00:51:29,930 things multiply but indices add. 813 00:51:29,930 --> 00:51:32,300 Things multiply, numbers multiply, 814 00:51:32,300 --> 00:51:34,160 while their indices add. 815 00:51:34,160 --> 00:51:35,990 That's the key idea of convolution 816 00:51:35,990 --> 00:51:40,410 that we'll see clearly and completely on Friday. 817 00:51:40,410 --> 00:51:41,660 OK.