Ryan Nelson on IT Project Management (Scaling Tech Podcast Ep5)

Aug 9, 2022 | Project Management

Professor Ryan Nelson joins us to discuss IT Project Management: Infamous Failures, Classic Mistakes, and Best Practices. Why are only 35% of projects successful? We discuss what he’s learned about project management over 30 years of research and case studies. Professor Nelson teaches at the University of Virginia’s McIntire School of Commerce, and is also Director of the Center for the Management of Information Technology. He discusses infamous failures like the FBI Trilogy Project, and how large organizations such as Capital One, CarMax, and the Washington Post have successfully adopted product mindsets and built in more flexible planning processes that help avoid the classic mistakes of IT Project Management.

Listen on Spotify
Listen on Apple Podcasts

Watch the video:
Show notes with links to jump ahead are below

Show Notes from Episode 5 – Ryan Nelson on IT Project Management
Timestamp links will open that part of the show in YouTube in a new window

  • 00:00 Arin and David start the show by talking about a major IT failure happening in Costa Rica right now, where the government tax system is down and businesses are unable to pay their taxes. This is a good example of the “infamous failures” that we’ll discuss in today’s episode.
  • 04:37 Arin introduces Professor Ryan Nelson. Professor Ryan Nelson teaches in the undergraduate, graduate, and executive education programs at the University of Virginia’s McIntire School of Commerce, where he has been a recipient of the All-University Teaching Award. He currently serves as the Director of the Center for the Management of Information Technology, a position he has held since the founding of CMIT in 1991. Professor Nelson has published articles and cases on project management, product management, and IT adoption in a variety of academic and professional journals, including MIS the MIS Quarterly Executive, the Communications of the ACM; and the Journal of Management Information Systems, just to name a few. He has also served as a consultant to a number of large corporation, including Bank of America, the Texas Air Corporation, Alere, and CFA Institute.
  • 06:19 Ryan starts by defining a project as a temporary endeavor with defined start and stop dates, with the goal of achieving a specific outcome. They have historically been the fundamental units of work organization.
  • 07:53 What is the cost of failure? Ryan’s students have studied many project failures over the last 22 years and come up with similar numbers as the Standish Group, which has measured that 2/3 of projects fail on either budget, time, or scope (“the Iron Triangle”). Those defined as “infamous failures” are the ones that get the most attention and have over $100 million USD in losses associated with the failure.
  • 10:10 Professor Nelson references Steve McConnell’s famous book “Rapid Development” which highlighted a set of 36 different classic mistakes that Ryan’s groups have built on and used as the foundation of their project retrospectives. The most common faiulres deal with estimation, risk management, poor planning, and stakeholder management failures. By studying these, we can hopefully prevent projects from failing at the same scale or frequency.
  • 11:55 David notes how amazing it is that there are trillions in losses per year in IT failures – covering private and public sectors. Professor Nelson notes that rough estimates indicated about $8 trillion USD are lost annually on a global scale, with about $2 trillion of that happening in the US.
  • 14:21 Professor Nelson noted a recent HBR article, “The Project Economy has arrived”, written by a former chair of the Project Management Institute, which references $20 trillion of project oriented work. But as David’s story about the tax issues in Costa Rica highlights, the impace is not just the losses of the money spent on the project itself, but the economic opportunity cost that the failure causes.
  • 15:54 One of the most infamous failures is the FBI’s Trilogy Project, which occurred around the time of the 9/11 attacks on the US. The mission of the FBI changed due to 9/11 and they became more of a terrorism focused organization. Their projects didn’t adapt to the mission change though, and so many of these ongoing projects ended up producing outputs that were no longer relevant, costing an estimated $170 million USD. When strategy changes, your projects must adapt to that change!
  • 18:14 The most innovative companies will do planning on a quarterly basis instead of annually, so that their plans are better adapted to strategic shifts. This has impacts on the communication cycles within companies and they need to be adapted to short cycles, and to be able to review the planning every few weeks. Even major companies like Capital One are able to do this to deliver more quickly.
  • 19:55 These shorter time cycles have driven companies like Capital One to change their mindset away from projects entirely and instead focus on product management. This has broken the cycle of annual budgets and helped them to manage more efficiently.
  • 21:00 David talks about the idea that poor risk management is a major cause of failure. One area he always looks for in project to mitigate risk is to look at every integration point. All software needs to be integrated with something else, and those integration points are the biggest areas of uncertainty, and the items that usually derail the overall effort estimation. Building software is “easy” compared with integrating it with the real world!
  • 23:49 Professor Nelson talks about how he’s observed the change in Silicon Valley companies, towards looking at Outcomes instead of Outputs. This comes down to Empowering and Inspiring your teams (referencing books by Marty Cagan, linked to below), which is really important in an Outcome driven organization which is focused on OKRs (Outcomes and Key Results). These OKRs become the North Star for teams, and the team can figure out what they need to do in order to achieve that.
  • 27:00 Arin comments how these projects that are failing are not just the ones we see in the news, but probably 2/3 of everyone’s projects. What are the changes we need to still enact to reduce these rates of failures? Professor Nelson talks about the importance of project retrospectives at a larger project scale, as well as the smaller sprint retrospectives that many agile methods call for. Even the daily standup is a form of a retrospective, and these help us to prevent the classic mistakes. What are the root causes of the classic mistakes that your organization is making? It’s not enough to be aware of them, but you need to find and address the root causes.
  • 29:45 What are some of the other best practices to resolve particular classic mistakes? Professor Nelson talks about a matrix to keep in mind of the best practices that align to classic mistakes and how you address them in your organization. Agile methods address many of these classic mistakes, by helping to address issues like estimation and communication. DevOps is another best practice to follow for better technical practices. Change management skills are a soft skill that’s very helpful also.
  • 32:00 Arin talks about how Agile helps to prevent 2/3 of projects from failing because Agile methods change our notion of success. Arin asks his client which item is most important to you? Is it Time, Budget, or Scope? Because you can’t have all 3 in most cases, and Agile methods enable that sort of conversation. The mindset that you cannot have all three is very important, if you acknowledge the other two are flexible, then that changes the definition of failure and success. Ryan agrees, and yet notes how many companies see these as “successful failure” because even though some of the metrics are missed, or a timeline or budget is missed, ultimately it was still successful because the outcomes achieved in the end were still valuable.
  • 35:00 Professor Nelson discusses a Project Model Canvas, which was described in the same HBR article mentioned earlier. What is the goal of this project? Let’s capture that up front. There are still some problems inherent to a project mindset, but at least it emphasizes the importance of determining a successful outcome first so the other metrics can be set aside.
  • 37:55 More universities are learning how to teach these mindsets, and offer not only a project management class but also a product management class. Ryan mentions Peter High’s book “Getting to Nimble”, which features case studies on a number of organizations and how they have adopted enterprise agility in companies like Capital One, CarMax, and the Washington Post. These groups are constantly re-evaluating the outcomes they need from a project or product and how they can adapt to changing needs.
  • 40:00 David talks about the potential consequences of growing a team, how to gradually add people so that the onboarding is not disruptive. Ryan talks about how organizations have accomplished digital transformations over the last decade. The Washington Post started with one small product team operating under the new practices, see how successful it is, why they are successful, and then try to scale more product oriented team after that. In one of his case studies, Ryan saw how CarMax went from one or two initial product teams, to then having 30 and then 50 different product teams. Then the challenge becomes how to communication organizational strategy across those teams so that they are all pointing in the right direction. CarMax does this with open houses every couple of weeks where product teams present what they have been doing recently and talk about any major challenges they are having. The CIO, COO and CMO are typically present to help communicate the strategy to these delivery teams.
  • 43:31 Professor Nelson talks about how organizations need to start reconstructing themselves to adapt to these new practices. It’s not just flipping a switch! Changes from Waterfall to Agile are not done overnight, and in large companies can be a decade long effort.
  • 45:11 Arin asks Ryan a controversial question based on another recent HBR article – “Do we still need teams?” The authors of that article described how hybrid work is making it very hard to manage teams and so maybe we should not use team structures in all situations. Professor Nelson and Arin both agree that we do in fact still need teams, and that high-functioning teams are very valuable. Ryan talks about how leading teams is not easy, but it’s how the best work is done. Working by yourself does not lead to the most diverse ideas, teamwork is needed.
  • 47:35 Professor Nelson discusses how the last couple of years in the pandemic have made it hard to have co-located teams, so the challenge now is how to make teams highly functional even when they are remote. Ryan mentions a case study they did on WillowTree apps, who had just finished a new office before the pandemic, and then were forced to decide what to do about co-location, which some of their pre-pandemic contracts required. This has been a huge challenge but they have found tools to help them collaborate remotely, and Professor Nelson’s students will be doing a follow up case study to see more about what they have learned during the pandemic.
  • 49:38 Teams are hard! But Professor Nelson doesn’t see a better way. Trying to become too task oriented and organizing to that detail just becomes like waterfall again, which is not going to work in the digital products world. Teams are still the way to go, even if remote work has made them harder to manage.
  • 52:10 Arin thanks Ryan for his work at the UVA McIntire School of Commerce, and for the influence that program has had on his career, and Professor Ryan shares a few kind words about Arin’s time there as a student as well. This episode was a “succesful success!”

Links from Episode 5 – Ryan Nelson on IT Project Management