Category Archives: Scaling

[Video 454] Ben Christensen: Don’t Build a Distributed Monolith

Is your Web application is getting a bit big and bulky on a single server? You might be thinking of moving to a more distributed, microservice-based architecture. And in many cases, that’s a great way to go about things. But microservices aren’t a panacea, having their own issues and pain points. What are the trade-offs associated with a microservice-based architecture? How do you define the dependencies (code and API) between the different services? And how can you avoid a “distributed monolith” — a situation in which you have many microservices that are dependent on one another? In this talk, Ben Christensen describes the different ways to think about such microservice- systems, and how to avoid some of the more common pitfalls.

[Video 447] Prasad Jonnalagadda: Content Delivery at Massive Scale using Control Plane Servers at Netflix

If you have a popular Web site, then you’ll realize (sooner or later) that the static content can and should be offloaded to a third-party service, known as a “content delivery network” (CDN), that specializes in getting content to users as quickly (and locally) as possible. Netflix also used CDNs for some time — but over the years, they have moved away from using third-party CDNs, and in favor of using their own, highly scaled, solution. Why and how did they make this change? How did this affect Netflix and its users? In this talk, Prasad Jonnalagadda describes the architecture that Netflix adopted, and the implications that this has had on the company and their technical teams.

Time: 1 hour

[Video 445] Barak Michener: Scaling Open Source Projects from 0 – 1000 Commits

A very large number of us use open-source software every day. Many also participate in open-source projects. But if you maintain an active open-source project, then you’re dealing with other people’s code on a very regular basis. How do you handle this? How do you manage a large number of commits, or pull requests, or bugs, or feature suggestions? When should you define, or enforce, code standards?  And what sorts of documentation standards shoudl you put in place?   In this talk, Barak Michener shares his experience running a number of open-source repositories and projects. If you are thinking of managing a new open-source project, or becoming an active participant on an existing project, then this talk will help you to consider how to scale your software — not in terms of efficiency, but rather in terms of people, community, and contributors.

[Video 431] Mandy Waite: Ok, so I have all these Containers, what now?

Containers are a big thing in IT and devops — the idea that you can have lightweight virtual machines that can be launched, duplicated, copied, and scaled up is very attractive, and for good reason. But what happens when you have a very large number of containers?  Have you simply exchanged one problem for another? Or are there methods and systems you can use to manage your containers? In this talk, Mandy Waite describes what she and others at Google are doing to make it easier to work with containers, and how you should start to look at containers if you’re to use them most effectively.

[Video 428] Guy L. Steele: Four Solutions to a Trivial Problem

So you want to add up a bunch of numbers, eh?  Fine, but what’s the fastest way in which you can do this?  What about if you want to parallelize your solution? What pitfalls will you encounter, and how can you avoid them?  In this talk, Guy L. Steele takes us, slowly but surely, though a number of different ideas, strategies, and concepts having to do with parallel programming, and considers the implications of them — regardless of the language you’re using.

[Video 412] Nik Everett: Wings Stay On, or Replacing On Site Search at Wikipedia

Wikipedia is both big and famous. Thus, any changes made to Wikipedia are going to affect many people. Moreover, something such as search is used frequently, by people around the world, and in many different languages. So, when the decision was made to change how Wikipedia implements its search system, this meant a lot of thought, planning, testing, and then deploying. In this talk, Nik Everett describes this process, and how Wikipedia switched over to using ElasticSearch without too many problems.


Time: 45m


[Video 402] Steven Proctor: Introduction to Erlang

Erlang is a functional language, designed for systems that cannot scale and that are highly parallelized. Why has interest in Erlang risen in the last few years, and why are people using a language that explicitly avoids state, objects, and threads in favor of functional techniques and processes? In this talk, Steven Proctor introduces Erlang, describing why it works the way it does, and how you can use it for better, faster, and more reliable software systems.

[Video 400] Colt McAnlis: The Hard Things about the Internet of Things

The “Internet of Things” is the modern embodiment of an idea that has been around for a while — that not only can people be interconnected on the network, but that our appliances, belongings, buildings, and even clothing can be interconnected on the network. The IoT is attracting a great deal of attention, but it’s also pointing to issues with the current infrastructure of the Internet, and with applications that will need to be solved. In this talk, Colt McAnlis points to the problems with implementing the Internet of Things, and how we might need to think about and solve these problems in order to allow our refrigerator, bathrobe, and skateboard to speak with one another.

[Video 399] Jeffrey Lembeck: Moneyball for Performance Metrics

The book (and movie) Moneyball described how it’s possible to use metrics and statistics to squeeze the greatest possible results out of a low-budget operation. In this talk, Jeffrey Lembeck tells us that we can and should do the same thing for our Web applications. How does this work?  What can (and should) we measure, and what should we do in response? If you want to do more with less (and all of us do), then you’ll want to watch this talk.

[Video 391] James Lewis: Building Systems that are #neverdone

It used to be so easy to set up an application: You set up a server, write some software, and then update the software over time.  But as systems get bigger and more complex, it becomes much harder to write, test, and deploy large, monolithic systems. Over time, we have moved toward microservices, which have their own challenges. In this talk, James Lewis describes our migration toward microservices, and what that means for designing, testing, deploying, and just thinking about such applications. What has changed, what has become easier, and what has become harder?