Host: Andre All righty, so yeah, I think that’s it. We’ve got plenty of people on the line, let’s crack into it. So this exclusive webinar, brought to you by Raygun and GitHub. We’re going to talk about speed and reliability, so how to move fast and fix things. Really looking forward to hearing from Sam and Nick about both their kind of unique perspectives on this topic. Let’s run through the agenda very quick, I’m going to introduce these two and hand it over to those guys shortly.

They’re going to run really quickly over what is Raygun and what is GitHub for those that aren’t aware, and then we’re going to jump into the discussion part of this topic. Once again, we’ve got the Q&A section and Zoom, so if you have any questions, please throw them in there. We’re going to get plenty of time for the Q&A. So we’ll move through the slides and then we’ll move to the Q&A. We also have some prizes, don’t we, Eliza?

Eliza Sure do.

Host: Andre So if you have questions, we’re going to pick the best question or two and we’ll send those best questions, the person who asked them, some swag from Raygun and GitHub. But first, I do have a quick poll, and this is kind of just to get a little bit of information about you guys so we can kind of tailor the conversation around where you are at in the journey, so let me launch that poll right now. Everyone can see that? So the first question is, “How many developers are there at your organization?” We’ve got a multiple choice option there. “How often does your team ship code into production?” That’s quite an interesting one, and then, “How much time does your team spend on bugs and performance issues?”

Host: Andre Just, you know, you’re not going to know that exactly, but roughly how much time does your time spend on bugs and performance issue. So I’m seeing the results fly in right now. So guys, we’ve got, what, 30% have got between 11 and 50 on their development teams, 45% between one and 10. Most people are shipping code weekly, which is pretty good. We’ve got 30% multiple times per day. And then people are spending 10 to 25% mostly on fixing and spending time on bugs and performance issues. Maybe I can kind of share results with you guys just very quickly, should be able to see that now. Yeah?

Sam Hunt We can see that.

Host: Andre Excellent, so that’s pretty interesting. We’ve got a bit of a mix here: Smaller teams between one and 10, [inaudible 00:05:04] actually up to 50 make up 70% of the audience today. We’ve got almost 20% with 100 or more, so that’s pretty cool. Multiple times per day, a third of our audience are shipping multiple times per day. That’s definitely where things are heading, and then 50% weekly, so that’s pretty good going as well. Wasn’t long ago where it far less regular than that. And then yeah, a vast majority are spending between 10 and 25% on bugs, so we appreciate that. All righty, so let’s move on from that and meet the presenters. We’ve got Sam Hunt, the VP of APAC at GitHub, and Nick Harley, the Director of Product. Also dabbles in a bit of marketing from Raygun. Guys, I’ll hand it over to you guys. Take it away. Sam, you want to kick it off?

Sam Hunt Cool, yeah. Hi, thanks for the intro. Sam Hunt, I run the APAC region for GitHub, working with customers, community on delivering the world’s largest code development platform, collaboration platform.

Nick Harley And I’m Nick, I’m the Director of Product, so my job’s more or less connecting with customers and finding out what we need to build, how we need to build it, and then wrangling developers and design teams to try and get those through the pipe as soon as possible.

Host: Andre And Nick, what is Raygun?

Nick Harley Yeah, so Raygun offers a suite of products that monitor your web and mobile applications, and then when errors, crashes, and performance issues affect users, we give developers all the diagnostic details they need to fix them with greater speed and accuracy.

Host: Andre Excellent, over to you Nick, I mean Sam.

Sam Hunt Yeah, so, I mean, for those that don’t know GitHub, as I mentioned it, it’s the world’s largest developer collaboration and code management platform. It’s where most of the world’s open source code lives. There’s 41 million developers collaborating on a daily basis on GitHub. But it’s also an extension into commercial collaboration around code as well, so open source is extremely relevant to everybody’s development ecosystem, but not everything we do is open source. We provide an extension which is the same tool for your commercial collaboration and development as well.

Host: Andre Awesome. Speed & Reliability: How to move fast and fix things is why we’re all here today. We’ll start off with Mentality & Culture, Nick.

Nick Harley Yeah, so this is quite an important point to get across: The actual inspiration for this webinar title kind of came from Mark Zuckerberg and Facebook, and Facebook’s a massive organization as we know, and we shouldn’t all go out and copy exactly what Facebook’s doing. But this kind of “Move Fast and Break Things” was a motto and a mantra that Facebook had throughout their offices several years ago, and it got picked up in the press, and lots of people kind of quote it, because it’s quite catchy.

Nick Harley But a lot of people don’t realize that this actually got superseded not long afterwards, and a quote here from Mark Zuckerberg at the Facebook developers conference, so that the idea is that developers moving quickly is so important that we are even willing to tolerate a few bugs in order to do that. What we realized over time is that wasn’t helping us to move faster because we had to slow down to fix these bugs, and it wasn’t improving our speed. So they actually pulled back from this “Move Fast and Break Things” to actually go with this less catchy “Move Fast with Stable Infrastructure” type model.

Nick Harley And if Facebook, with all their resources, and money, and such a big company can’t actually pull off moving fast and fixing things, it kind of points to more of this problem. You know, I think Facebook in the early days could get away with it, because the company’s on a rapid growth trajectory, and it’s just way better for developers to actually fix things as they go along. And every company now is becoming a software company, so what do all these companies have in common? Well, they didn’t start out as software companies. They’ve had to grow into software, and better monitoring, and application performance is kind of the core aspect of what they do nowadays.

Nick Harley Because high street retailers have mobile apps, websites to manage, stock ordering systems. Even places like Home Depot, you can now order online. They’re now an online-first company, and if we don’t have good software experiences, people are just going to shop next door. And so right in the middle of there, Dominoes pizza, I mean, they’re a pizza company, retail. But if we actually look at their share price over time, they just … They use technology, if you’ve ever seen Dominoes pizza and what they come out with, with pizza trackers, there’s a pizza checker thing being advertised at the moment.

Nick Harley And if all of this technology didn’t actually work very well for users, then this share price would look a lot different. So developers are actually in this unique position at the moment where the faster they can build, it’s actually a market opportunity and a business outcome, that developers are actually empowering these companies to do well.

Host: Andre It’s interesting that the developers are in the center of this, aren’t they? The center of innovation for traditionally non-technical companies.

Sam Hunt Yeah, absolutely, and Nick, just to extend on one of your examples, look at the automotive industry. We all consume cars at some point in our lives, or most of us do, and from where the car industry competed 10 to 15 years ago, it was around colors, and wheel sizes, and speed, and gearboxes. And the reality is most of us go out and buy a car based on the technology that’s going into it. So there’s an industry that’s completely … It’s still very connected to its roots of being a manufacturing industry, but it’s completely shifted to how it competes to being a technology industry as well.

Nick Harley Cool, and if we move on to mentality within a team over monitoring, it doesn’t really matter whether you’ve got a New Relic, a Raygun, a [inaudible 00:11:13], whatever tool you’re using to monitor your software, you can buy a tool, but it won’t actually do anything for your team if you don’t use it correctly or have the team culture around monitoring of software quality. So when you are moving fast, you can actually spot issues and fix them as you go along. Just buying a tool is really irrelevant unless you use it properly.

Nick Harley And a lot of the mistakes that we see with development teams is they talk a lot about, “Oh, how many errors does my application have? How many problems does our application, our software, our infrastructure …” It’s very kind of internally focused, and we really need to break out of this kind of like, “It’s about our application, and our software, and our code,” because it’s really not. The outcomes are that it’s affecting your customers, and your end user experience, and your conversion rates, and the revenue of the business, and these are all outcomes.

Nick Harley The things in the middle are all tactics, you can choose what to monitor, you can choose which parts of your application that you need to pay attention to. But the outcomes are about your customers, and who really actually cares the most about these outcomes? It’s usually the CEO, the senior management, the executive team, the head of product. All the people in your organization that are more senior and you want to impress care about these outcomes. So when we’re actually looking at being able to communicate these things within our teams, it’s best to focus on these outcomes. It’s like the biggest career hack you can have, just to take what you’re doing at the code level and then talk about them more at an outcome level and how it affects the business.

Nick Harley One of the key measures that I look for in a team when I ask engineers is do they actually look forward to shipping new features, or does it kind of fill them with this sense of dread? Like, “Ah, if we push this live it’s going to actually go wrong, and there’s no real safety net.” So if you can actually get your team to think about this, do we have confidence in shipping what we’ve built? Have we got the monitoring in place to make sure if it does go wrong then we can pull it back? That’s kind of the litmus test of have we actually got this culture and this team mentality to the right place?

Nick Harley And deploying with confidence really comes from how will we actually deploy code safely as a team, and how can we increase the frequency to several times a day? And continuous integration and continuous deployment are essential for kind of quick development cycles. So this is kind of the first step we need to get to. On the poll there, people are at kind of mostly weekly deployments, but is there an appetite in the business to make that daily or even improve it just from where you are?

Nick Harley And if you can get there, things like code reviews, pair programming, bug bash Fridays, all these types of things can actually help you get your software quality improved, and the speed that you are shipping software can improve as well. And if we put deployment and release tracking around issues, we can actually ship with confidence, because we know that we’re going to catch them once they go out into production. So gone are the days where we used to have all our kind of changes rolled up in one big release, and we released it on the shipping day, and everyone would get really nervous around it. If we can actually increase the frequency that we ship, we’ll actually see that have huge benefits across the business, because things are going out in smaller pieces.

Sam Hunt Nick, can I just … I really like this, and I think it’s key, and I’m going to talk a little bit more about this. I’m obviously not so much from the technical side but more from the culture side, but this really, really draws to the point that tool’s a part of your team, and getting the feedback from them in context of what you’re trying to deliver as an outcome based on your previous slide is a really, really important part of trusting that whole ecosystem that’s delivering. I mean, the reality of the reason why we can deliver so much code now is because we’re able to trust and automate with the great tools that we use, the parts that just become the bog down of your day to day. And I think it’s a really important point.

Nick Harley Yeah, that’s exactly right. It has to start with your team and your culture, and the mentality you have around this stuff. So how can you actually improve from where you are today to ship faster and with better quality?

Sam Hunt And you’re kind of comparing your team to the best in the world these days, with people’s expectations and [inaudible 00:15:55] Netflix, or Slack.

Nick Harley Yeah.

Sam Hunt And so you’ve kind of got to be looking at the best of the world and kind of moving towards that directional big part of that journey, which is changing so quickly.

Nick Harley Yeah. I mean, I read something a while ago around Amazon shipping multiple times a day, hundreds of times, hundreds of times. But I thought I’d include this example of how we do it at Raygun, just to give people a perspective of our … We could always do better obviously, but we feel as though we’ve got quite a good process going on in this area. And so we use Jira for our issue tracking and prioritizing work. And then once that goes into the dev team, they use their own kind of tooling. We don’t have too many kind of restrictions on what they use, but things like Rider and Visual Studio are pretty prevalent within our team.

Nick Harley And then we use GitHub to manage our source code. And when people commit their code, we run automated continuous integration builds, which will fail if there’s a problem in the application and people can fix. And then we use Octopus Deploy to actually deploy our code to beta environments, test environments, and promote it to production when it’s ready. But while it’s on the beta environment and the integration tests are passed, we’ll get other people on the team to code review everything that will go out the door. And these changes they need to make will go through that process again, but if it’s okay we’ll merge that into the master, run that through the build again, and then we obviously put it into production, and we have Raygun watching for alerts.

Nick Harley We tag the deployments. With Raygun, there’s a deployment tracking functionality, so you can actually see which issues you’ve introduced, and we have that hooked up to Slack as well. So we know if we were to ship something to production, if we start getting errors in our Slack channel, then we know we can roll it back pretty quickly. But hopefully that doesn’t happen because of the QA, and the integration tests, and the safety nets that have been put in place. And then as we do have issues in production, because nobody’s perfect, we can … Well, we can deploy pretty easily, and we can roll back pretty easily.

Nick Harley So with Octopus Deploy, we usually run it on one of the office environments and then promote it into a beta, which is as close to production as possible, before promoting it to production. So we’re actually in this situation where we can roll back deployments within seconds, and we can ship to production within around five minutes. The biggest thing that takes the most amount of time is actually the build server in that process, to actually build the code. And if people are looking to get a bit more information on this, Ollie on our team is on our dev team, and could probably speak to it more than me. At this Bitly link at the bottom, there’s a blog post there that goes into detail about how we actually ship software like this.

Nick Harley And so the last point on this culture section is you’ve signed up to this webinar because you have an interest in actually moving faster and fixing things as you go along. But you really need to kind of stand out in your organization and actually make this happen. So I’ve spoken to multiple people who have actually had success with driving this culture within their team, and they always feel like they were a little bit going out on a limb initially. They felt like they didn’t have the buy-in. But when they spoke about these business outcomes and they tied it all back to outcome-based approaches, they actually got the buy-in of the executive team, and were able to kind of accelerate their own career as well as improve the organization in their engineering team.

Nick Harley Because they stuck their neck out and say, “Hey, well, I’m not comfortable with what we’re doing right now. We can do better.” So I think you know, it is going to accelerate your career and get you more respect if you just try and do this continuous improvement in your team, and the end result is better software for your customers. So there’s no real downside to wanting to improve software velocity and quality at the same time.

Sam Hunt Yeah, so I think sort of extending on that from the culture perspective, which is really where my expertise more comes into it than the technology side of things, there’s really three key areas that we want to focus on in this. And the first one, we’ll cover them off, they all have slightly different tangents, but the first one is really getting over the fear factor around your work. And that’s where there’s relevance around how tools can help you do that, but also how people can help you do that as well. But the work without fear is shifting as well, and it means different things to different people.

Sam Hunt You can see on your screen right now, you see a lion. That lion, kind of for me, that’s the 20th century, that’s the CEO of an organization. They work without fear, they own the risk, and they make decisions. But the reality of the 21st century is we’re not lions. We’re honey badgers, right? Honey badgers work without fear. We have to be in there, we have to be aggressive, and there’s two things that really help us do that: One is to be comfortable with having lots of eyes on things and collaborating on our mistakes and our wins as well, and the other is relying on tools. And this is a great shift in the way that businesses see the value of technology as well.

Sam Hunt So that lion’s still there, but that lion now respects what the honey badger means to their organization, because let’s face it, we talked about every company being a technology company a little earlier, it couldn’t be truer. And this, the importance of what you’re doing at a development level resonates all the way up. So if we take that, how do we make that relevant? It’s around collaboration. So we talk about collaboration within our own developer community and our own teams, but it’s also collaboration with the business. It’s really, really important.

Sam Hunt And as I said before, collaboration with tools. Tools are part of your business as well. So the relevance to the information that they’re giving back to you has to be taken in the same way that you would take feedback from a colleague that might be doing a code review, or you’ve asked a question for, and it’s all about how those pieces collaborate together to bring outcomes that are in line with what we’re all trying to achieve. Now, here’s the hard bit: We’re all trying to achieve little different things. So if we go and look at what a developer wants, well, the reality is they kind of want chaos.

Sam Hunt They want to try a lot of things, they want to play, they want to come out with innovative outcomes, but business don’t always want that, right? They want some control there, they want to mitigate risk and make sure that there’s some sort of boundaries. So this is where we talk about finding a balance between your control and your chaos, and as you see the railroad tracks on the screen, it’s kind of like the business owns one side and technology owns the other. And you need to collaborate and come to agreement on where your rails sit and how wide they are or how narrow they are.

Sam Hunt Now, the downside is you can get really, really tight on the control, but that narrows your ability to innovate and ship at speed. And then there’s a downside to being really, really open, because you lose a lot of the control. The interesting fact is, it’s usually your consumer or your industry that’s going to dictate where those rails should be. So the natural balance here is where you have the rails in a position where you can compete and stay up to date with what the market’s asking you to do. So I’d ask yourself to take a step back and look. If you feel like you’re falling behind your competitors or your industry demand, your rails are probably too close.

Sam Hunt If you feel like you have trust issues with your consumer because of product stability or inconsistencies, your rails are probably too wide. And it’s really, really important, pulling it back into the collaboration piece, because we haven’t always been good at having business talk to technology and coming out with aligned outcomes, and that’s what’s going to help us succeed. It’s not just a developer decision, it’s not just a business decision. It’s a company decision for outcomes, and remembering that those outcomes, you have measureables in the market where you can figure out what’s important, what am I missing at? It’s not hard to do.

Sam Hunt Tools like Raygun, they really give you an opportunity to have another insight, and it gives you, in addition to that, it gives you great insight at speed as well, and that’s the important thing. So I think if we can focus on those three points, working without fear, and again, that can mean a lot of things: It could be fear of criticism is one that I actually hear a lot, developers being worried that somebody’s going to put eyes on their code and criticize it. Be conscious of how you criticize, be constructive in how you criticize. Because really, you want that.

Sam Hunt We all improve better by getting constructive criticism back, and it is a cultural shift away to be comfortable with that. That ties into the collaboration, so really, really important that collaboration is a key part of our strategy moving forward. And again, as I said, it’s not just collaboration within ourselves. It’s collaboration within our industry, it’s collaboration within open source communities. We’re probably all leveraging open source technology that helps us deliver, because we don’t want to reinvent the wheel. And then the last one is identifying the boundaries and those rails, and how we’re going to work closely together with aligned outcomes that are going to allow us to develop at the speed that we have to to either compete or stay relevant.

Host: Andre That’s some healthy kind of conflict there, isn’t it, between business objectives and being safe to what basically your customer demands as fast innovation, not affecting the performance or the experience.

Sam Hunt Yeah, absolutely, and I think people forget that your product’s being consumed by something, someone. And it’s what they think of it as much as anything else. And if we look at disruptors, so the Airbnbs, the Ubers, the Netflix of the world, they changed industries. And if you didn’t stay relevant, you disappeared.

Eliza: Guest host They’ve changed the industries, but they’ve also changed our expectations.

Sam Hunt Yeah.

Eliza: Guest host I know a lot of governments are finding it difficult because the way that they operate doesn’t really allow, but we expect apps to be able to change deeds on houses and things like that now. So consumer expectations have certainly shifted.

Sam Hunt Yeah, I actually, I was on a panel in Sydney yesterday. I was talking to Eliza about this earlier, and just before I went on the panel my daughter emailed me from her science class asking me to send her a design for a mousetrap [inaudible 00:27:25], right? Her expectation is that I’m available right now. Assume that your consumer is in the exact same position.

Host: Andre I mean, it’s changing so quickly, you know? That’s constantly moving.

Nick Harley Yeah, so we talked a lot about kind of technology and teams, but how do you actually get the buy-in to some of this stuff? And for me, I am a personal fan of dashboards, because having dashboards up in the office is just really powerful. People know what’s going on, things like that. But what I’ve encountered across multiple companies is the dashboards just show the success metrics: “How many sales did we make? How many goal completions did we have?” Things like that. But it takes a very long time for those metrics to start sagging before anybody notices, “Oh, there’s actually something going on under the hood here.”

Nick Harley So I’m a big believer in having really raw metrics of like, “Tell me when this stuff is broken on the dashboard in the office.” And what does your team actually need to pay attention to? So have this discussion with your team, “What do we need to monitor?” We actually have this concept of canaries here at Raygun where we have these kind of signals, like, “Nobody can pay, nobody can do this and that.” And thankfully, these canaries don’t go off very often, but we’ve got them there to tell us straight away when something’s broken. So how can you put up dashboards in your office that kind of tell you when things are going wrong so that somebody doesn’t have to go a month later and go, “Why are sales down so much this month?” and you have a checkout issue or something like that.

Sam Hunt Yeah, well again, it comes back to tools being part of the team.

Nick Harley Yep.