1 00:00:00,090 --> 00:00:02,430 The following content is provided under a Creative 2 00:00:02,430 --> 00:00:03,820 Commons license. 3 00:00:03,820 --> 00:00:06,030 Your support will help MIT Open CourseWare 4 00:00:06,030 --> 00:00:10,120 continue to offer high-quality educational resources for free. 5 00:00:10,120 --> 00:00:12,660 To make a donation or to view additional materials 6 00:00:12,660 --> 00:00:16,620 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:16,620 --> 00:00:20,420 at ocw.mit.edu 8 00:00:20,420 --> 00:00:22,170 WILLIAM BONVILLIAN: Rasheed, why don't you 9 00:00:22,170 --> 00:00:29,080 give us that point that you were making to me about DARPA. 10 00:00:29,080 --> 00:00:33,670 RASHEED: Yes, so DARPA relies very heavily on this program 11 00:00:33,670 --> 00:00:34,540 manager structure. 12 00:00:34,540 --> 00:00:36,220 So they give these program managers a lot of power 13 00:00:36,220 --> 00:00:38,320 and a lot of leeway to do whatever they want. 14 00:00:38,320 --> 00:00:41,950 But they have-- they have to have a very distinct vision 15 00:00:41,950 --> 00:00:43,840 from the outset, like really when they 16 00:00:43,840 --> 00:00:45,540 start to be able to do things. 17 00:00:45,540 --> 00:00:49,658 And so I was concerned that these program managers were 18 00:00:49,658 --> 00:00:51,700 going to basically be coming from the same places 19 00:00:51,700 --> 00:00:54,605 over and over again, and so have very similar visions. 20 00:00:54,605 --> 00:00:56,230 And you're going to be able to miss out 21 00:00:56,230 --> 00:00:57,760 on a lot of different talent that 22 00:00:57,760 --> 00:01:00,040 exists outside, or a lot of different new projects, 23 00:01:00,040 --> 00:01:03,220 or new fields, maybe, just like you're talking 24 00:01:03,220 --> 00:01:06,490 about-- these new collaboration between biology 25 00:01:06,490 --> 00:01:10,000 and kind of humans and computer science, things like that. 26 00:01:10,000 --> 00:01:11,860 But that was really my main concern, 27 00:01:11,860 --> 00:01:17,570 was program managers weren't being chosen as-- 28 00:01:17,570 --> 00:01:20,290 equitably isn't the word, but that the selection 29 00:01:20,290 --> 00:01:22,767 process of these program managers was actually-- 30 00:01:22,767 --> 00:01:23,600 might be too narrow. 31 00:01:26,170 --> 00:01:29,280 WILLIAM BONVILLIAN: But they do get visionaries. 32 00:01:29,280 --> 00:01:30,488 STUDENT: Sometimes. 33 00:01:30,488 --> 00:01:32,280 I mean, I think we mentioned that they only 34 00:01:32,280 --> 00:01:35,338 got white males [INAUDIBLE]. 35 00:01:39,040 --> 00:01:40,540 WILLIAM BONVILLIAN: OK, so I'm going 36 00:01:40,540 --> 00:01:42,940 to move us to Tammy Carleton. 37 00:01:42,940 --> 00:01:47,500 And Tammy's big perception about DARPA 38 00:01:47,500 --> 00:01:53,890 is this vision perception, that that's a critical factor that 39 00:01:53,890 --> 00:01:55,780 makes DARPA different. 40 00:01:55,780 --> 00:01:57,940 And there is a process that DARPA 41 00:01:57,940 --> 00:02:02,920 has developed for this radical technological visioning. 42 00:02:02,920 --> 00:02:06,880 And essentially, she argues that there's six stages that they 43 00:02:06,880 --> 00:02:08,139 use to kind of get there. 44 00:02:08,139 --> 00:02:11,050 First, recruitment of great talent-- 45 00:02:11,050 --> 00:02:13,600 and the talent, they hear about talent 46 00:02:13,600 --> 00:02:16,870 from this DARPA network, which is now quite huge. 47 00:02:16,870 --> 00:02:19,720 Because all of the people that get DARPA contracts and awards, 48 00:02:19,720 --> 00:02:22,180 they stay in this kind of DARPA family. 49 00:02:22,180 --> 00:02:24,460 And typically, researchers love to work for DARPA. 50 00:02:24,460 --> 00:02:31,480 Because working on DARPA-hard problems is really intriguing. 51 00:02:31,480 --> 00:02:34,780 So that's a community that tends to know each other, 52 00:02:34,780 --> 00:02:36,370 and be in communication, as well as 53 00:02:36,370 --> 00:02:40,920 the community of former DARPA program managers and officials. 54 00:02:40,920 --> 00:02:44,800 So a lot of the talent comes through that network, 55 00:02:44,800 --> 00:02:49,090 which kind of knows what's going to work. 56 00:02:49,090 --> 00:02:52,120 So the recruitment and the vision get united in a way, 57 00:02:52,120 --> 00:02:54,160 in that recruitment process. 58 00:02:54,160 --> 00:02:57,402 Then comes the stage of the vision formulation. 59 00:02:57,402 --> 00:02:59,110 The program manager has got to figure out 60 00:02:59,110 --> 00:03:02,080 what they're going to do with their four or five years. 61 00:03:02,080 --> 00:03:06,920 And they've got to formulate a radical vision, 62 00:03:06,920 --> 00:03:08,800 a right/left vision, what's going to come out 63 00:03:08,800 --> 00:03:11,217 of that pipeline, and what's the fundamental research that 64 00:03:11,217 --> 00:03:13,450 going to get me there. 65 00:03:13,450 --> 00:03:16,570 And it's driving for that breakthrough that's 66 00:03:16,570 --> 00:03:19,330 the organizing proposition. 67 00:03:19,330 --> 00:03:21,010 Then, they've got to go through defining 68 00:03:21,010 --> 00:03:23,260 a program and its launch. 69 00:03:23,260 --> 00:03:26,500 Often, they will have a workshop, 70 00:03:26,500 --> 00:03:28,090 an invitation-only workshop, where 71 00:03:28,090 --> 00:03:34,270 they'll assemble 25, 30, 40 of the kind of best thinkers 72 00:03:34,270 --> 00:03:38,320 on a project area, and meet for a day or two days. 73 00:03:38,320 --> 00:03:44,620 I've watched one of those, which got held up at MIT as part 74 00:03:44,620 --> 00:03:47,980 of the beginnings of creating this Biological Technologies 75 00:03:47,980 --> 00:03:49,000 office. 76 00:03:49,000 --> 00:03:52,210 And they brought in just an amazing collection, 77 00:03:52,210 --> 00:03:54,940 from MIT and from elsewhere, of thinkers 78 00:03:54,940 --> 00:03:57,040 about what some of the early projects 79 00:03:57,040 --> 00:04:00,630 might be to achieve some of these capabilities. 80 00:04:00,630 --> 00:04:03,520 So the Deputy Director of DARPA came up, 81 00:04:03,520 --> 00:04:05,202 and kind of helped run this workshop 82 00:04:05,202 --> 00:04:06,160 and think this through. 83 00:04:06,160 --> 00:04:07,870 The office wasn't set up for a while 84 00:04:07,870 --> 00:04:10,540 until the next DARPA director, but it really 85 00:04:10,540 --> 00:04:13,720 helped contribute to that portfolio. 86 00:04:13,720 --> 00:04:18,190 And it was a really serious effort. 87 00:04:18,190 --> 00:04:23,320 So the program manager develops a portfolio approach 88 00:04:23,320 --> 00:04:25,090 of standing up a series of groups 89 00:04:25,090 --> 00:04:28,090 that are going to be working on this, and inter-relating them. 90 00:04:28,090 --> 00:04:29,940 And the program manager is also going 91 00:04:29,940 --> 00:04:33,635 to be responsible for making this technology transfer. 92 00:04:33,635 --> 00:04:35,260 So in other words, it's not enough just 93 00:04:35,260 --> 00:04:36,190 to do something cool. 94 00:04:36,190 --> 00:04:38,800 You've got to move it into the implementation stage. 95 00:04:38,800 --> 00:04:40,520 And how is that going to happen? 96 00:04:40,520 --> 00:04:43,300 So the program manager has got to undertake 97 00:04:43,300 --> 00:04:46,810 all of these various steps. 98 00:04:46,810 --> 00:04:51,220 So the criteria for being DARPA hard 99 00:04:51,220 --> 00:04:54,460 is that the project has to be technologically very 100 00:04:54,460 --> 00:04:55,160 challenging. 101 00:04:55,160 --> 00:04:58,120 It's got to extend beyond the current limits of technology 102 00:04:58,120 --> 00:04:59,050 and knowledge. 103 00:04:59,050 --> 00:05:01,900 Otherwise, they don't really want to do it. 104 00:05:01,900 --> 00:05:04,280 It's got to be actionable, which means it can be made. 105 00:05:04,280 --> 00:05:06,670 It can be built. It can be produced. 106 00:05:06,670 --> 00:05:09,580 So if it can't be, why do it? 107 00:05:09,580 --> 00:05:11,080 You've got to be able to demonstrate 108 00:05:11,080 --> 00:05:12,940 that it's actionable. 109 00:05:12,940 --> 00:05:14,860 Typically multidisciplinary-- that it's 110 00:05:14,860 --> 00:05:17,800 going to draw on a variety of areas of expertise. 111 00:05:17,800 --> 00:05:19,000 It's got to be far reaching. 112 00:05:19,000 --> 00:05:20,650 It's got to change things. 113 00:05:20,650 --> 00:05:24,640 It's got to be very ambitious at a big scale. 114 00:05:24,640 --> 00:05:27,280 "Don't do little things" is kind of a DARPA sub-rule. 115 00:05:29,830 --> 00:05:32,980 And various techniques to flesh out the vision 116 00:05:32,980 --> 00:05:35,050 include those expert-only workshops, 117 00:05:35,050 --> 00:05:38,170 and a proof of concept effort. 118 00:05:38,170 --> 00:05:40,450 Sometimes, if it's some idea that looks promising, 119 00:05:40,450 --> 00:05:42,880 they'll do what DARPA calls a "seedling," just put 120 00:05:42,880 --> 00:05:45,190 a little money on it, and let somebody play 121 00:05:45,190 --> 00:05:47,680 with it, a real thinker play with it, 122 00:05:47,680 --> 00:05:50,380 and see what it might materialize into. 123 00:05:50,380 --> 00:05:53,080 So they've got that seedling capability. 124 00:05:53,080 --> 00:05:57,138 There is no training system for program managers. 125 00:05:57,138 --> 00:05:58,930 They don't go through any kind of training. 126 00:05:58,930 --> 00:06:00,880 They just arrive. 127 00:06:00,880 --> 00:06:05,133 And they kind of learn from other program managers 128 00:06:05,133 --> 00:06:06,300 what they're supposed to do. 129 00:06:06,300 --> 00:06:10,390 So there's a very strong kind of DARPA culture 130 00:06:10,390 --> 00:06:13,600 about how it operates, that gets communicated by word of mouth 131 00:06:13,600 --> 00:06:16,870 and kind of learning by doing. 132 00:06:16,870 --> 00:06:23,260 And it's informal exchange, but there is a very strong kind 133 00:06:23,260 --> 00:06:26,170 of operating rules set. 134 00:06:26,170 --> 00:06:28,060 DARPA program managers really have 135 00:06:28,060 --> 00:06:29,900 to be quite entrepreneurial. 136 00:06:29,900 --> 00:06:32,470 And Martine, you were raising this earlier. 137 00:06:32,470 --> 00:06:34,720 They've got to stand up a whole new territory. 138 00:06:34,720 --> 00:06:36,550 And to do that, they've got to be 139 00:06:36,550 --> 00:06:41,440 highly entrepreneurial about how to bring that about. 140 00:06:41,440 --> 00:06:45,030 And the decision to stand up a vision program-- 141 00:06:45,030 --> 00:06:48,100 this is not set up through peer review. 142 00:06:48,100 --> 00:06:50,860 There isn't consensus decision-making at DARPA. 143 00:06:50,860 --> 00:06:54,850 That program manager has got to have a really strong vision, 144 00:06:54,850 --> 00:06:58,060 and it's got to get really organized. 145 00:06:58,060 --> 00:07:01,600 And then it's got to withstand the office director 146 00:07:01,600 --> 00:07:06,310 and then the DARPA director really tearing through it, 147 00:07:06,310 --> 00:07:08,110 and really analyzing it. 148 00:07:08,110 --> 00:07:10,390 So once you get that, though, the approvals 149 00:07:10,390 --> 00:07:13,000 can be very fast, once the decision is 150 00:07:13,000 --> 00:07:15,160 made to stand something up. 151 00:07:15,160 --> 00:07:17,590 And that the program manager really is-- 152 00:07:17,590 --> 00:07:21,080 to go back to that term that you liked, Steph-- 153 00:07:21,080 --> 00:07:23,380 a vision champion. 154 00:07:23,380 --> 00:07:26,600 They've got to champion this vision and make it happen. 155 00:07:26,600 --> 00:07:29,200 That's what the program manager has got to do. 156 00:07:29,200 --> 00:07:34,780 So DARPA starts with division. 157 00:07:34,780 --> 00:07:40,820 That is not how industry operates. 158 00:07:40,820 --> 00:07:44,968 That's not how other government agencies operate. 159 00:07:49,570 --> 00:07:54,880 Industry works largely on what's called a "stage gate system." 160 00:07:54,880 --> 00:07:57,730 So industry is going to want to do typically 161 00:07:57,730 --> 00:07:59,410 more incremental kind of advances, 162 00:07:59,410 --> 00:08:02,320 because the advances can snap onto markets that it's already 163 00:08:02,320 --> 00:08:05,560 got, and economic opportunities that it already sees, 164 00:08:05,560 --> 00:08:08,710 and technologies that it's already starting to explore. 165 00:08:08,710 --> 00:08:10,300 So it's typically going to want to do 166 00:08:10,300 --> 00:08:12,850 more incremental advances. 167 00:08:12,850 --> 00:08:18,060 And it's only going to do those incremental advances if they 168 00:08:18,060 --> 00:08:21,070 can optimize them economically. 169 00:08:21,070 --> 00:08:23,650 So an industry R&D project has got 170 00:08:23,650 --> 00:08:29,400 to go through a series of stage gates, which essentially 171 00:08:29,400 --> 00:08:34,890 is a way of tearing down a large menu of stuff 172 00:08:34,890 --> 00:08:39,270 and getting to the most economically optimal. 173 00:08:39,270 --> 00:08:42,580 Industry stage gate process is not a technology visioning 174 00:08:42,580 --> 00:08:43,080 system. 175 00:08:45,940 --> 00:08:48,582 It's not deciding on a vision and doing 176 00:08:48,582 --> 00:08:50,290 whatever is necessary to get there, which 177 00:08:50,290 --> 00:08:52,000 is the way DARPA operates. 178 00:08:52,000 --> 00:08:55,240 It's a much more limited set of procedural stage gates 179 00:08:55,240 --> 00:08:57,670 that a technology idea has got to go through. 180 00:08:57,670 --> 00:09:01,570 So DARPA just working on a very different kind 181 00:09:01,570 --> 00:09:04,390 of organizational model from the way in which things are done, 182 00:09:04,390 --> 00:09:13,030 either at other agencies or within industry. 183 00:09:13,030 --> 00:09:15,100 And starting with this vision up front 184 00:09:15,100 --> 00:09:18,850 is a really interesting organizational idea, 185 00:09:18,850 --> 00:09:22,270 that you all should kind of keep in mind when you're 186 00:09:22,270 --> 00:09:24,820 operating out there, and setting up your own companies, 187 00:09:24,820 --> 00:09:27,490 and doing your own startups. 188 00:09:27,490 --> 00:09:30,450 So Max, you've got Tammy Carleton's piece. 189 00:09:30,450 --> 00:09:31,240 MAX: Right. 190 00:09:31,240 --> 00:09:36,622 The key step within Carleton's five steps 191 00:09:36,622 --> 00:09:38,080 that I thought was most interesting 192 00:09:38,080 --> 00:09:40,060 was the fifth one, technology transfer, which 193 00:09:40,060 --> 00:09:44,200 is supposed to bridge the R&D-- 194 00:09:44,200 --> 00:09:47,680 the valley of death, as it were, that 195 00:09:47,680 --> 00:09:52,280 enables a different scientific idea to be commercialized, 196 00:09:52,280 --> 00:09:57,220 and to actually affect people all over the world, eventually. 197 00:09:57,220 --> 00:09:59,980 While reading, I kind of realized 198 00:09:59,980 --> 00:10:03,020 that DARPA seems like they have a pretty decent organization. 199 00:10:03,020 --> 00:10:05,260 They get things done relatively efficiently. 200 00:10:05,260 --> 00:10:07,220 They have a lot of money. 201 00:10:07,220 --> 00:10:08,970 Why is it they have-- 202 00:10:08,970 --> 00:10:11,230 and I actually just decided to start googling, 203 00:10:11,230 --> 00:10:13,750 to try to figure out if DARPA had done anything related 204 00:10:13,750 --> 00:10:14,260 to fusion. 205 00:10:14,260 --> 00:10:16,090 And I couldn't find anything. 206 00:10:16,090 --> 00:10:20,492 So I'm curious, why not? 207 00:10:20,492 --> 00:10:21,950 WILLIAM BONVILLIAN: Well, actually, 208 00:10:21,950 --> 00:10:24,730 DARPA deferred on energy technologies 209 00:10:24,730 --> 00:10:28,030 to ARPA-E, when ARPA-E got stood up, 210 00:10:28,030 --> 00:10:32,680 because DARPA views ARPA-E as a compatriot. 211 00:10:32,680 --> 00:10:35,860 And ARPA-E interestingly, has done some extremely interesting 212 00:10:35,860 --> 00:10:38,992 work on fusion. 213 00:10:38,992 --> 00:10:41,590 STUDENT: I know Lockheed [INAUDIBLE] fusion-- 214 00:10:41,590 --> 00:10:42,673 WILLIAM BONVILLIAN: Right. 215 00:10:42,673 --> 00:10:43,630 STUDENT: [INAUDIBLE] 216 00:10:43,630 --> 00:10:47,243 MAX: Yeah, but Lockheed Martin's-- 217 00:10:47,243 --> 00:10:49,410 WILLIAM BONVILLIAN: Well, they have the Skunk Works. 218 00:10:49,410 --> 00:10:50,050 MAX: They do. 219 00:10:50,050 --> 00:10:52,675 WILLIAM BONVILLIAN: And DARPA's done an enormous amount of work 220 00:10:52,675 --> 00:10:56,013 with the Skunk Works over time, including stealth. 221 00:10:56,013 --> 00:10:58,180 MAX: Granted, yes-- they definitely have a facility, 222 00:10:58,180 --> 00:11:00,880 and they definitely have the project managers. 223 00:11:00,880 --> 00:11:02,620 I just feel that it's a bit of an area 224 00:11:02,620 --> 00:11:04,150 outside of their expertise. 225 00:11:04,150 --> 00:11:06,657 They're going to accidentally make it fly and stuff. 226 00:11:06,657 --> 00:11:07,740 WILLIAM BONVILLIAN: Right. 227 00:11:07,740 --> 00:11:09,790 And look-- another issue for DARPA is-- remember, 228 00:11:09,790 --> 00:11:11,582 these technologies have got to get stood up 229 00:11:11,582 --> 00:11:14,213 in the lifetime of the project. 230 00:11:14,213 --> 00:11:15,130 MAX: Yeah, five years. 231 00:11:15,130 --> 00:11:16,370 WILLIAM BONVILLIAN: Like a five year project. 232 00:11:16,370 --> 00:11:18,040 So if you can't do it within five years, 233 00:11:18,040 --> 00:11:20,410 it's very hard to put it on the DARPA agenda. 234 00:11:20,410 --> 00:11:22,832 MAX: Well, you mentioned that some of the projects 235 00:11:22,832 --> 00:11:24,790 would get handed off to other project managers. 236 00:11:24,790 --> 00:11:25,120 WILLIAM BONVILLIAN: Yes. 237 00:11:25,120 --> 00:11:27,630 MAX: So that's how long a manager lasts, right? 238 00:11:27,630 --> 00:11:28,630 WILLIAM BONVILLIAN: Yes. 239 00:11:28,630 --> 00:11:30,790 MAX: But it's not always how long a project lasts. 240 00:11:30,790 --> 00:11:31,790 WILLIAM BONVILLIAN: Yes. 241 00:11:31,790 --> 00:11:34,690 Sometimes, there can be intergenerational transfers. 242 00:11:34,690 --> 00:11:38,620 And then Licklider developed a different kind of system. 243 00:11:38,620 --> 00:11:44,080 He created a community that kept continuing 244 00:11:44,080 --> 00:11:45,880 with the same kind of ethos, working 245 00:11:45,880 --> 00:11:47,770 on that same set of agenda items, 246 00:11:47,770 --> 00:11:50,200 working them down one after another, 247 00:11:50,200 --> 00:11:53,205 and making progress year after year on each of them. 248 00:11:53,205 --> 00:11:54,580 So that's a different kind of way 249 00:11:54,580 --> 00:11:56,080 that DARPA is going to be organized. 250 00:11:56,080 --> 00:11:59,530 But typically, once the program manager 251 00:11:59,530 --> 00:12:03,250 leaves who had the vision, their job 252 00:12:03,250 --> 00:12:05,790 is to get that implemented in their, in effect, 253 00:12:05,790 --> 00:12:07,240 DARPA lifetime. 254 00:12:07,240 --> 00:12:10,150 And if they haven't, then it's hard for DARPA 255 00:12:10,150 --> 00:12:13,810 to continue that, which we could view 256 00:12:13,810 --> 00:12:16,525 as a weakness of the program. 257 00:12:16,525 --> 00:12:18,650 On the other hand, you could view it as a strength. 258 00:12:18,650 --> 00:12:20,817 MAX: Yeah, because then they make sure that things-- 259 00:12:20,817 --> 00:12:22,600 that if they are working, then they keep-- 260 00:12:22,600 --> 00:12:24,250 they may proceed with it, or they pass it off 261 00:12:24,250 --> 00:12:25,090 to someone else. 262 00:12:25,090 --> 00:12:26,090 WILLIAM BONVILLIAN: Yes. 263 00:12:30,570 --> 00:12:33,990 MAX: So speaking of Licklider, actually, while I was reading, 264 00:12:33,990 --> 00:12:37,980 I noticed that Licklider had mentioned that he heavily 265 00:12:37,980 --> 00:12:40,480 relied on standardized tests when picking candidates, 266 00:12:40,480 --> 00:12:43,440 so like that recruitment of great talent. 267 00:12:43,440 --> 00:12:45,300 Given that literally everyone in this room 268 00:12:45,300 --> 00:12:48,270 has experience with standardized testing, and at least 269 00:12:48,270 --> 00:12:51,840 I can't speak for everyone, but I don't like it very much. 270 00:12:51,840 --> 00:12:52,550 I wonder-- 271 00:12:52,550 --> 00:12:53,842 STUDENT: [INAUDIBLE] two tests. 272 00:12:53,842 --> 00:12:54,690 MAX: Which two? 273 00:12:54,690 --> 00:12:55,350 It was ACTs. 274 00:12:55,350 --> 00:12:56,490 STUDENT: One was like a psych test, 275 00:12:56,490 --> 00:12:57,626 and the other one was like a [INAUDIBLE].. 276 00:12:57,626 --> 00:12:58,320 STUDENT 1: GRE. 277 00:12:58,320 --> 00:12:59,742 We did the GRE, it said. 278 00:12:59,742 --> 00:13:01,200 STUDENT: Well, there's another one, 279 00:13:01,200 --> 00:13:02,742 but it seemed like a psychology test. 280 00:13:02,742 --> 00:13:04,013 I'm not sure. 281 00:13:04,013 --> 00:13:05,680 STUDENT 1: So I think that she mentioned 282 00:13:05,680 --> 00:13:08,800 that he was the only one who asked for test scores at all. 283 00:13:08,800 --> 00:13:12,997 And one was the GRE, which is kind of weird. 284 00:13:12,997 --> 00:13:15,580 WILLIAM BONVILLIAN: Remember, he was a psychologist coming out 285 00:13:15,580 --> 00:13:16,930 of the testing world. 286 00:13:16,930 --> 00:13:21,060 So that's probably part of the reason. 287 00:13:21,060 --> 00:13:25,270 MAX: I'll chalk it up to [INAUDIBLE],, because there's 288 00:13:25,270 --> 00:13:27,187 only so much a standardized test can tell you. 289 00:13:27,187 --> 00:13:29,353 They won't really tell you how you work with people. 290 00:13:29,353 --> 00:13:30,970 They won't tell you anything about how 291 00:13:30,970 --> 00:13:33,580 you work on anything that's not related to, I don't know, 292 00:13:33,580 --> 00:13:36,205 math or literature. 293 00:13:36,205 --> 00:13:38,830 WILLIAM BONVILLIAN: He did come up with a pretty talented team, 294 00:13:38,830 --> 00:13:39,140 though. 295 00:13:39,140 --> 00:13:40,098 MAX: Yeah, that's true. 296 00:13:40,098 --> 00:13:42,870 STUDENT: I mean, you get pretty far on psychology tests, 297 00:13:42,870 --> 00:13:44,460 especially when it comes to intertia. 298 00:13:44,460 --> 00:13:46,835 Because you have to figure out, for really good visionary 299 00:13:46,835 --> 00:13:49,560 leaders, what they'll figure out is 300 00:13:49,560 --> 00:13:52,040 they're intellectuals, introverts, extroverts, 301 00:13:52,040 --> 00:13:55,800 and they have a supporting side to their psyche. 302 00:13:55,800 --> 00:13:58,560 And they're kind of like the avatar of team groups. 303 00:13:58,560 --> 00:14:02,190 So it's pretty good for if you want to find out good leaders. 304 00:14:02,190 --> 00:14:04,964 And I know it's been pretty effective in manageent theory 305 00:14:04,964 --> 00:14:06,862 when you use psych tests. 306 00:14:06,862 --> 00:14:08,320 It's also really good to figure out 307 00:14:08,320 --> 00:14:10,535 who is going to be a really good scientist, 308 00:14:10,535 --> 00:14:12,160 because it's usually highly introverted 309 00:14:12,160 --> 00:14:16,120 and comes up with new insights that doesn't follow the crowd. 310 00:14:16,120 --> 00:14:17,830 So I can see why that's important. 311 00:14:17,830 --> 00:14:20,840 The GRE might be, but also, if your score is low, 312 00:14:20,840 --> 00:14:22,967 but if you've come out with a good body of work, 313 00:14:22,967 --> 00:14:24,550 then they can probably assume that you 314 00:14:24,550 --> 00:14:26,383 are pretty smart, even if you didn't do well 315 00:14:26,383 --> 00:14:27,331 on standardized tests. 316 00:14:29,902 --> 00:14:31,860 STUDENT 2: Because I'm looking up her LinkedIn. 317 00:14:31,860 --> 00:14:35,080 And it says that she has-- her background is in communication. 318 00:14:35,080 --> 00:14:37,080 And then she got a Master's in Public Relations. 319 00:14:37,080 --> 00:14:39,625 And then she did a PhD in Mechanical Engineering Design. 320 00:14:39,625 --> 00:14:42,000 But that must have been back in the day when the D School 321 00:14:42,000 --> 00:14:45,090 at Stanford accepted non-engineers 322 00:14:45,090 --> 00:14:47,820 and non-scientists, because now they don't. 323 00:14:47,820 --> 00:14:51,840 So it would be curious to think about the ways in which 324 00:14:51,840 --> 00:14:54,750 her background, in particular, might give her that insight 325 00:14:54,750 --> 00:14:57,870 that an engineer might not have about this particular field, 326 00:14:57,870 --> 00:15:01,080 going back to the question of multidisciplinary great groups. 327 00:15:01,080 --> 00:15:03,070 But I don't know to what extent, for example, 328 00:15:03,070 --> 00:15:05,840 someone like me might be accepted in maybe Lilly's lab. 329 00:15:05,840 --> 00:15:08,250 LILLY: One of the most famous women in my field 330 00:15:08,250 --> 00:15:13,500 has an undergraduate degree in German. 331 00:15:13,500 --> 00:15:14,840 So you never know. 332 00:15:14,840 --> 00:15:19,340 STUDENT: Lloyd Blankfein, the CEO of J P Morgan, I think-- 333 00:15:19,340 --> 00:15:20,490 his degree's in history. 334 00:15:20,490 --> 00:15:21,975 And he almost barely didn't pass. 335 00:15:21,975 --> 00:15:23,350 He said he still gets nightmares, 336 00:15:23,350 --> 00:15:26,965 because he didn't fulfill all the requirements. 337 00:15:26,965 --> 00:15:28,590 WILLIAM BONVILLIAN: All right, so we're 338 00:15:28,590 --> 00:15:31,410 going to move out of the testing world back to the DARPA world. 339 00:15:31,410 --> 00:15:33,870 MAX: Sure. 340 00:15:33,870 --> 00:15:36,420 So one of the things that Carleton pointed out 341 00:15:36,420 --> 00:15:42,510 was that as DARPA has aged, so, too, have its project managers. 342 00:15:42,510 --> 00:15:46,710 Even though they cycle between new managers every four or five 343 00:15:46,710 --> 00:15:50,820 years, the average age of the managers that they would pick 344 00:15:50,820 --> 00:15:53,610 was going up steadily. 345 00:15:53,610 --> 00:15:55,200 So I'm curious if anyone has thoughts 346 00:15:55,200 --> 00:15:58,230 on what the effect of that would be on their managerial ability, 347 00:15:58,230 --> 00:16:02,280 or on DARPA's direction as a whole. 348 00:16:02,280 --> 00:16:04,030 STUDENT: Could you repeat that first part? 349 00:16:04,030 --> 00:16:04,530 MAX: Yeah. 350 00:16:04,530 --> 00:16:07,080 So the average age of the project managers 351 00:16:07,080 --> 00:16:08,807 has been increasing, as DARPA has-- 352 00:16:08,807 --> 00:16:10,140 STUDENT: What's the average age? 353 00:16:10,140 --> 00:16:11,557 MAX: I don't remember the numbers. 354 00:16:11,557 --> 00:16:12,680 STUDENT: Is it 27? 355 00:16:12,680 --> 00:16:14,240 WILLIAM BONVILLIAN: No, it's a lot older than that. 356 00:16:14,240 --> 00:16:15,323 STUDENT: It's a lot older? 357 00:16:15,323 --> 00:16:16,820 MAX: Yeah. 358 00:16:16,820 --> 00:16:20,180 WILLIAM BONVILLIAN: One of the reasons, Max, 359 00:16:20,180 --> 00:16:22,760 may be that it's-- 360 00:16:26,230 --> 00:16:31,070 I mean, watching MIT, it's very hard for younger faculty 361 00:16:31,070 --> 00:16:34,460 to walk out of the place before they get tenure. 362 00:16:34,460 --> 00:16:37,430 It's hard enough to walk out once you get tenure, 363 00:16:37,430 --> 00:16:41,330 but it's impossible to walk out before you get tenure. 364 00:16:41,330 --> 00:16:43,710 And because you're establishing a whole-- 365 00:16:43,710 --> 00:16:46,730 your research portfolio and your whole researching system, 366 00:16:46,730 --> 00:16:49,080 and it's very hard to walk out of that. 367 00:16:49,080 --> 00:16:52,835 In addition, when you go to DARPA, you've got a conflict. 368 00:16:52,835 --> 00:16:54,710 Once you've worked there, you have a conflict 369 00:16:54,710 --> 00:16:55,970 of interest with DARPA. 370 00:16:55,970 --> 00:16:57,428 You're not going to be able to deal 371 00:16:57,428 --> 00:17:00,270 with the agency for a certain period of time. 372 00:17:00,270 --> 00:17:02,478 So if they happen to be funding your research before, 373 00:17:02,478 --> 00:17:05,062 they're not going to be able to continue funding your research 374 00:17:05,062 --> 00:17:06,480 once you've worked there. 375 00:17:06,480 --> 00:17:08,480 So it's gotten more complicated for DARPA 376 00:17:08,480 --> 00:17:10,760 to recruit at the university level. 377 00:17:10,760 --> 00:17:12,210 It's gotten a lot more difficult. 378 00:17:12,210 --> 00:17:14,660 So the route between MIT and DARPA, 379 00:17:14,660 --> 00:17:17,480 frankly, used to be a lot easier to manage. 380 00:17:17,480 --> 00:17:23,460 And when I look at my colleagues here who have gone to DARPA, 381 00:17:23,460 --> 00:17:25,988 they are more typically fairly far along, 382 00:17:25,988 --> 00:17:28,280 and very assured about their career, and their research 383 00:17:28,280 --> 00:17:29,530 foundation, and their writing. 384 00:17:29,530 --> 00:17:31,340 They've done that stuff, and they 385 00:17:31,340 --> 00:17:33,890 can take a three or five year break, 386 00:17:33,890 --> 00:17:36,830 and get that really big, visionary thing 387 00:17:36,830 --> 00:17:39,800 that they've wanted to work on forever done 388 00:17:39,800 --> 00:17:40,730 when they go to DARPA. 389 00:17:40,730 --> 00:17:42,730 But it's harder when you're trying to administer 390 00:17:42,730 --> 00:17:44,930 a whole research portfolio. 391 00:17:44,930 --> 00:17:47,230 Again, with the IPA authority, you've 392 00:17:47,230 --> 00:17:49,760 got a certain freedom to come back to your organization 393 00:17:49,760 --> 00:17:51,530 and keep the day-to-day stuff going. 394 00:17:51,530 --> 00:17:53,240 But it's still not simple. 395 00:17:53,240 --> 00:17:57,140 So that may be one part of the reason 396 00:17:57,140 --> 00:18:00,170 that you tend to get somewhat older faculty than they 397 00:18:00,170 --> 00:18:01,832 probably used to at the beginning. 398 00:18:01,832 --> 00:18:03,838 STUDENT 1: Licklider wasn't especially young 399 00:18:03,838 --> 00:18:04,880 when he was the director. 400 00:18:04,880 --> 00:18:06,710 WILLIAM BONVILLIAN: Right, he wasn't young. 401 00:18:06,710 --> 00:18:08,380 STUDENT 1: Or the project manager. 402 00:18:08,380 --> 00:18:11,940 So I think age does affect the perspective that you bring in. 403 00:18:11,940 --> 00:18:15,710 But having older people isn't a bad thing. 404 00:18:15,710 --> 00:18:17,610 I think in the tech world, it's seen 405 00:18:17,610 --> 00:18:20,590 as like if you're not young and hip, 406 00:18:20,590 --> 00:18:23,440 you're going to be out of touch with what's going to work. 407 00:18:23,440 --> 00:18:24,770 But I don't know if that's a-- 408 00:18:24,770 --> 00:18:26,240 I think the points that Bill made 409 00:18:26,240 --> 00:18:28,598 about having to kind of establish yourself and figure 410 00:18:28,598 --> 00:18:30,890 out what your vision is, you have more time when you're 411 00:18:30,890 --> 00:18:33,730 older to have a vision. 412 00:18:33,730 --> 00:18:35,470 LILLY: Yeah, and perspective as well. 413 00:18:35,470 --> 00:18:38,620 Just thinking about advisors I've had over the years, 414 00:18:38,620 --> 00:18:41,800 and younger or newer advisors versus more 415 00:18:41,800 --> 00:18:43,570 established advisors-- they definitely 416 00:18:43,570 --> 00:18:47,810 have a different perspective. 417 00:18:47,810 --> 00:18:53,290 And yeah, NASA is actually having the same phenomenon, 418 00:18:53,290 --> 00:18:57,690 with having directors who are progressively older and older. 419 00:18:57,690 --> 00:19:00,360 So that's very sim-- they can't-- 420 00:19:00,360 --> 00:19:02,940 it's very difficult to draw someone away 421 00:19:02,940 --> 00:19:06,210 from a mid-range career if they're being very successful, 422 00:19:06,210 --> 00:19:07,780 and having success getting grants. 423 00:19:07,780 --> 00:19:12,470 And it's hard to get them into administration or directorships 424 00:19:12,470 --> 00:19:13,053 or management. 425 00:19:13,053 --> 00:19:14,470 STUDENT: What would the difference 426 00:19:14,470 --> 00:19:16,125 between young and older directors be? 427 00:19:20,780 --> 00:19:25,170 LILLY: In general, actually, the more established and older 428 00:19:25,170 --> 00:19:28,710 advisors that I've had have had a lot more 429 00:19:28,710 --> 00:19:35,409 perspicacity with respect to what projects are worthwhile, 430 00:19:35,409 --> 00:19:39,467 actually, or-- 431 00:19:39,467 --> 00:19:41,300 WILLIAM BONVILLIAN: Through long experience, 432 00:19:41,300 --> 00:19:44,780 you're suggesting, because they've seen a lot. 433 00:19:44,902 --> 00:19:46,610 LILLY: Yeah, that's the impression I get. 434 00:19:46,610 --> 00:19:47,693 WILLIAM BONVILLIAN: Right. 435 00:19:49,630 --> 00:19:51,010 Next, how about another question? 436 00:19:51,010 --> 00:19:51,728 MAX: Sure. 437 00:19:51,728 --> 00:19:52,664 All right. 438 00:19:58,280 --> 00:20:02,750 So Carleton mentioned that DARPA, 439 00:20:02,750 --> 00:20:05,222 in order to save time when they are trying 440 00:20:05,222 --> 00:20:06,680 to develop these projects, they try 441 00:20:06,680 --> 00:20:10,040 to discourage consensus, which I didn't really see as a-- 442 00:20:10,040 --> 00:20:13,050 I think it was the second, the next slide you have. 443 00:20:13,050 --> 00:20:13,550 Yeah. 444 00:20:13,550 --> 00:20:17,362 So I understand why, because then it 445 00:20:17,362 --> 00:20:19,320 ensures that whatever you're trying to develop, 446 00:20:19,320 --> 00:20:20,630 it gets done quickly. 447 00:20:20,630 --> 00:20:22,370 But just because it gets done quickly 448 00:20:22,370 --> 00:20:24,410 doesn't really mean that it gets done right. 449 00:20:24,410 --> 00:20:28,820 So I guess I was curious what people thought on that, 450 00:20:28,820 --> 00:20:31,028 specifically that decision. 451 00:20:31,028 --> 00:20:33,320 WILLIAM BONVILLIAN: It's a very interesting point, Max. 452 00:20:33,320 --> 00:20:35,750 And actually, ARPA-E-- and we'll talk about it 453 00:20:35,750 --> 00:20:38,770 when we get to ARPA-E and the energy class, 454 00:20:38,770 --> 00:20:42,390 but ARPA-E does have a process. 455 00:20:42,390 --> 00:20:46,010 It's a much smaller agency, so it's 300 million, not 3 billion 456 00:20:46,010 --> 00:20:47,120 like DARPA. 457 00:20:47,120 --> 00:20:52,370 So it's like a big DARPA office, sizable DARPA office, 458 00:20:52,370 --> 00:20:54,290 one of its five or six offices. 459 00:20:54,290 --> 00:20:58,280 So ARPA-E does have a consensus process. 460 00:20:58,280 --> 00:21:01,730 They bring in-- they call them "project directors" 461 00:21:01,730 --> 00:21:03,830 but they're program managers. 462 00:21:03,830 --> 00:21:05,660 And that community, really, you have 463 00:21:05,660 --> 00:21:08,900 to present your vision to that community. 464 00:21:08,900 --> 00:21:12,890 And that's a very tough-minded group, and a very sophisticated 465 00:21:12,890 --> 00:21:13,700 group. 466 00:21:13,700 --> 00:21:17,150 And the director is quite sophisticated at ARPA-E 467 00:21:17,150 --> 00:21:18,530 historically, as well. 468 00:21:18,530 --> 00:21:20,900 So they did a variation on this. 469 00:21:20,900 --> 00:21:24,680 At DARPA, it's much more, convince your office director, 470 00:21:24,680 --> 00:21:27,950 and then the office director will work with you 471 00:21:27,950 --> 00:21:29,660 on convincing the director of DARPA. 472 00:21:29,660 --> 00:21:32,330 It's a simpler, more straightforward process. 473 00:21:32,330 --> 00:21:33,980 You do get support from other program 474 00:21:33,980 --> 00:21:36,870 managers, who will have a lot of advice and ideas for you. 475 00:21:36,870 --> 00:21:39,813 So it's a pretty supportive operation. 476 00:21:39,813 --> 00:21:41,480 Nobody's in competition with each other. 477 00:21:41,480 --> 00:21:42,855 They're there to help each other, 478 00:21:42,855 --> 00:21:46,970 so that's a positive, although again, they're 479 00:21:46,970 --> 00:21:48,530 competing to achieve their vision. 480 00:21:48,530 --> 00:21:49,905 And everybody kind of knows who's 481 00:21:49,905 --> 00:21:52,362 getting their vision done, and kind of who is 482 00:21:52,362 --> 00:21:53,570 in trouble with their vision. 483 00:21:53,570 --> 00:21:55,640 So that's kind of known and understood. 484 00:21:55,640 --> 00:21:58,070 There's a certain kind of competition in that way, 485 00:21:58,070 --> 00:22:01,860 but it's not a direct one-on-one competition with each other. 486 00:22:01,860 --> 00:22:05,030 But at ARPA-E, there is more of a consensus decision making. 487 00:22:05,030 --> 00:22:10,970 And Luyao when she was telling us about Xerox PARC, 488 00:22:10,970 --> 00:22:15,140 remember, your whole discussion about the dealer process 489 00:22:15,140 --> 00:22:18,440 and having to bring your idea to the whole team sitting around 490 00:22:18,440 --> 00:22:21,050 on beanbags, who would tear it apart-- 491 00:22:21,050 --> 00:22:24,590 that's kind of a consensus process. 492 00:22:24,590 --> 00:22:27,740 It's a definite, tough-minded review process 493 00:22:27,740 --> 00:22:30,087 that DARPA doesn't really quite have. 494 00:22:30,087 --> 00:22:32,420 So there may be strengths and weaknesses in both models. 495 00:22:37,668 --> 00:22:38,960 Max, what do you think on that? 496 00:22:42,040 --> 00:22:44,990 MAX: Yeah, I could see there'd be strengths and weaknesses 497 00:22:44,990 --> 00:22:45,490 to both. 498 00:22:45,490 --> 00:22:54,083 But it just feels if you have less consensus, than you have-- 499 00:22:54,083 --> 00:22:55,750 if you have fewer people that are trying 500 00:22:55,750 --> 00:22:57,640 to discuss these ideas, then there 501 00:22:57,640 --> 00:22:59,960 might be some critical flaw that you might not-- 502 00:22:59,960 --> 00:23:02,502 that you might just miss, just because there are fewer people 503 00:23:02,502 --> 00:23:06,670 with more diverse experiences. 504 00:23:06,670 --> 00:23:10,240 Of course, if you have a decent idea of the science behind it, 505 00:23:10,240 --> 00:23:12,220 and it's not something as untested 506 00:23:12,220 --> 00:23:15,370 as fusion, for example, then I could 507 00:23:15,370 --> 00:23:17,210 see it working pretty well. 508 00:23:17,210 --> 00:23:20,980 I mean, if you had to do that for-- 509 00:23:20,980 --> 00:23:24,220 I could see it having problems with something untested 510 00:23:24,220 --> 00:23:25,630 like stealth. 511 00:23:25,630 --> 00:23:27,640 But apparently, they made it work. 512 00:23:27,640 --> 00:23:34,888 So maybe they're passing my expectations. 513 00:23:34,888 --> 00:23:38,890 STUDENT: [INAUDIBLE] Silicon Valley perspective, 514 00:23:38,890 --> 00:23:42,520 like it's a pretty common mantra to say 515 00:23:42,520 --> 00:23:44,980 you want to have an idea that people kind of say 516 00:23:44,980 --> 00:23:46,690 is not a good idea. 517 00:23:46,690 --> 00:23:48,550 Because the idea is, if you're coming up 518 00:23:48,550 --> 00:23:49,900 with something that's pretty innovative, 519 00:23:49,900 --> 00:23:51,220 most likely people won't agree with it, 520 00:23:51,220 --> 00:23:52,900 or it doesn't make sense, or you have an insight 521 00:23:52,900 --> 00:23:54,245 that other people don't get. 522 00:23:54,245 --> 00:23:55,870 Also, if you're an expert, you probably 523 00:23:55,870 --> 00:23:58,020 know some things that other people don't know, 524 00:23:58,020 --> 00:24:00,687 that you'd have to bring them up to speed on, especially if it's 525 00:24:00,687 --> 00:24:02,520 in a completely new area. 526 00:24:02,520 --> 00:24:05,582 And so the history has been, they're 527 00:24:05,582 --> 00:24:08,825 just saying that for new fields, who is the expert? 528 00:24:08,825 --> 00:24:10,450 It wouldn't be a PhD, because the field 529 00:24:10,450 --> 00:24:11,880 hasn't been invented yet. 530 00:24:11,880 --> 00:24:15,360 It's this person who just started working on it. 531 00:24:15,360 --> 00:24:20,100 So Bill Gates was an expert on software for personal computers 532 00:24:20,100 --> 00:24:21,860 before that even became a thing. 533 00:24:21,860 --> 00:24:23,400 So it's very common to-- consensus 534 00:24:23,400 --> 00:24:26,940 is actually really bad, because you put down these ideas based 535 00:24:26,940 --> 00:24:27,730 on new insights. 536 00:24:27,730 --> 00:24:28,230 So. 537 00:24:28,230 --> 00:24:30,240 LILLY: Yeah, and another issue with consensus 538 00:24:30,240 --> 00:24:32,670 is you have to assume, or you have 539 00:24:32,670 --> 00:24:37,800 to have a group in which all members have the personality 540 00:24:37,800 --> 00:24:43,380 type that they will concede to someone else's idea 541 00:24:43,380 --> 00:24:45,620 to get consensus, even if it's not 542 00:24:45,620 --> 00:24:47,415 their pet project or their favorite. 543 00:24:47,415 --> 00:24:48,290 You know what I mean? 544 00:24:48,290 --> 00:24:50,978 Some people aren't disposed to do that, no matter what. 545 00:24:53,935 --> 00:24:56,310 WILLIAM BONVILLIAN: About a closing thought on this, Max? 546 00:24:56,310 --> 00:24:57,460 MAX: Sure. 547 00:24:57,460 --> 00:25:01,600 So overall, I feel that DARPA seems 548 00:25:01,600 --> 00:25:03,890 to have been a pretty great success. 549 00:25:03,890 --> 00:25:06,070 They've made lots of really cool projects. 550 00:25:06,070 --> 00:25:08,170 They do it very quickly. 551 00:25:08,170 --> 00:25:12,550 Not sure how cheaply they do it, but they get it done. 552 00:25:12,550 --> 00:25:14,500 And because of that continued success, 553 00:25:14,500 --> 00:25:18,580 I feel that that's probably why they continue 554 00:25:18,580 --> 00:25:21,250 to get so much funding. 555 00:25:21,250 --> 00:25:25,840 I really try to emphasize that technology transfer aspect, 556 00:25:25,840 --> 00:25:29,740 because it ensures that the technology that they develop 557 00:25:29,740 --> 00:25:33,190 isn't just it's a nice, new toy for the military to have. 558 00:25:33,190 --> 00:25:35,830 It ensures that whatever they make, from the internet, 559 00:25:35,830 --> 00:25:38,990 GPS, et cetera, that everyone can use it, 560 00:25:38,990 --> 00:25:42,310 and that it makes our entire society better. 561 00:25:42,310 --> 00:25:48,890 So I guess that's pretty much just why I like DARPA. 562 00:25:48,890 --> 00:25:50,150 WILLIAM BONVILLIAN: Great. 563 00:25:50,150 --> 00:25:53,500 There's a lot of MIT folks and people saying, what do you do? 564 00:25:53,500 --> 00:25:58,180 All right, let me do a quick wrap up of today's class. 565 00:25:58,180 --> 00:26:01,990 So we did Mitch Waldrop's book. 566 00:26:01,990 --> 00:26:04,210 And I really wanted to kind of portray 567 00:26:04,210 --> 00:26:07,500 the governmental role in supporting the earlier stage 568 00:26:07,500 --> 00:26:09,820 of the evolution of computing. 569 00:26:09,820 --> 00:26:13,420 It was a proving ground for new concepts, 570 00:26:13,420 --> 00:26:17,200 and designs, and architectures. 571 00:26:17,200 --> 00:26:19,270 The Defense Department created an initial market 572 00:26:19,270 --> 00:26:21,440 for a lot of the new products, and services, 573 00:26:21,440 --> 00:26:24,070 and in fact, whole industries. 574 00:26:24,070 --> 00:26:27,580 This greatly expanded university research capabilities, 575 00:26:27,580 --> 00:26:31,190 the computing revolution. 576 00:26:31,190 --> 00:26:36,920 And it was done to further governmental missions. 577 00:26:36,920 --> 00:26:40,950 But it had, obviously, dramatic societal effects. 578 00:26:40,950 --> 00:26:43,910 So it featured governmental agencies 579 00:26:43,910 --> 00:26:45,800 playing a pretty central role. 580 00:26:45,800 --> 00:26:47,670 DARPA, as we discussed was critical, 581 00:26:47,670 --> 00:26:51,230 but some of the others were necessary and needed. 582 00:26:51,230 --> 00:26:53,300 And the governmental agency sponsored 583 00:26:53,300 --> 00:26:56,990 these industry-university collaborations 584 00:26:56,990 --> 00:27:00,050 that led to great strength in the university side, 585 00:27:00,050 --> 00:27:02,330 and focus areas around them. 586 00:27:02,330 --> 00:27:07,990 So in the reading that we did from the textbook, 587 00:27:07,990 --> 00:27:09,740 the central point here was that DARPA 588 00:27:09,740 --> 00:27:14,600 was uniquely able to combine an innovation institutional role, 589 00:27:14,600 --> 00:27:17,150 as well as a great groups innovation 590 00:27:17,150 --> 00:27:21,470 at the face-to-face level kind of role. 591 00:27:21,470 --> 00:27:22,940 That's a remarkable accomplishment. 592 00:27:22,940 --> 00:27:28,370 And then DARPA's ability, as we talked about in that reading 593 00:27:28,370 --> 00:27:34,070 as well, to innovate in a legacy sector, 594 00:27:34,070 --> 00:27:37,610 to do a frontier sector like IT, but also 595 00:27:37,610 --> 00:27:41,060 innovate within a legacy sector, and do things like stealth, 596 00:27:41,060 --> 00:27:42,560 and UAVs-- 597 00:27:42,560 --> 00:27:46,670 that's a pretty fascinating organization, 598 00:27:46,670 --> 00:27:48,920 that's flexible, and interesting, 599 00:27:48,920 --> 00:27:51,140 and able to do great stuff. 600 00:27:51,140 --> 00:27:53,450 Glenn Fong's piece was really about DARPA 601 00:27:53,450 --> 00:27:55,820 playing the central institutional mobilization 602 00:27:55,820 --> 00:27:58,340 role for the IT revolution. 603 00:27:58,340 --> 00:28:01,160 And Tammy Carleton's piece really 604 00:28:01,160 --> 00:28:04,070 told us about this kind of central importance 605 00:28:04,070 --> 00:28:09,980 of technology visioning as a key modus operandi, that DARPA 606 00:28:09,980 --> 00:28:11,712 been able to operate in. 607 00:28:11,712 --> 00:28:14,762 Any closing questions? 608 00:28:14,762 --> 00:28:16,220 Good luck with your paper outlines. 609 00:28:16,220 --> 00:28:18,170 Let me know if you have questions. 610 00:28:18,170 --> 00:28:20,950 And I look forward to seeing you on Tuesday.