Atlanta, GA, USA
2020
  |  By Matt LeRay
Handwritten mocks are cheap one at a time. This series built enough of them to show how quickly that stops being true. Nine posts took one package notifier from a function returning "delayed" to a captured response from a real carrier. Along the way, we hand-authored canned successes, failure cases, a spy, a stateful fake, an HTTP server, response fixtures, and contract-drift tests in four languages.
  |  By Matt LeRay
The notifier has returned a message throughout this series, which made testing almost suspiciously easy. Assert on the return value and you are done. Real notifiers do more than build strings: they send them. Once a message goes to an email provider or SMS gateway, the function may return nothing useful. When that change lands, every existing test loses the value it asserted on. This is part 4 of a ten-part series. The code is in Java, Node.js, Go and Python.
  |  By Matt LeRay
Every codebase has a failure path nobody has run. Not through laziness, but because reproducing it requires a backend dependency to misbehave on cue. In the package notifier, the carrier must refuse, stall, or return nonsense at the exact moment the test runs. So the retry logic ships unverified and everyone hopes. The seam from post 2 already gives the test control. A seam is a place where you can change what code does without editing that code.
  |  By Matt LeRay
The spy from post 4 is a sharp tool. Once a test can record every interaction, it is tempting to assert on all of them. The result looks thorough, but it is usually a transcript rather than a useful specification. This post takes a test written that way, makes a change that no customer could possibly notice, and watches the test fail anyway. This is part 5 of a ten-part series. The code is in Java, Node.js, Go and Python.
  |  By Alan Mon
The dashboards are red, the alerts are firing, and Slack has officially become a war room. Someone has asked, “Anyone else seeing this?” and 14 people have immediately responded with screenshots. Welcome to another day in distributed systems. The recent GitHub outage is a great reminder of how complicated modern applications have become. Services talk to services, infrastructure scales up and down, retries multiply traffic, and dependencies behave in ways nobody expected.
  |  By Matt LeRay
Private cloud and on-premises infrastructure are different. I know this. I used the terms interchangeably anyway because I was trying to compress a pattern into one sentence. That was sloppy, but the underlying pattern still matters. Banks run Kubernetes differently from most other organizations. Not because every bank makes the same choices, but because similar constraints can produce a similar pattern.
  |  By Matt LeRay
Somewhere in your service is a branch that has never executed. Not a rare one, a never one. It handles a dependency being unavailable: it reads from a cache, it returns a stale value, it marks the response degraded so callers know not to trust it too far. It was written carefully. It was reviewed. Whether it works is an open question, because nothing in the test suite makes that dependency fail, and the dependency does not fail on request.
  |  By Matt LeRay
Scoped chaos rules are now in proxymock. A filter query picks the traffic, an effect perturbs it, and every response that gets touched is labelled so you can tell an injected failure from a real one for the rest of the run. Available in v2.5.892 and newer. The short version of why: kill a pod and you learn something real, but you do not learn what your service does when a dependency stays up and starts lying to it.
  |  By Ken Ahrens
An engineering leader asked me a question a few weeks ago: could we read their business workflows out of API traffic instead of asking people to document them? I said it should be possible. Then I tried it. A few engineers know how the system really works. They know which calls make up a work order and which checks happen after a write. That stuff rarely makes it into the test plan. Usually it’s in somebody’s head. Sometimes it’s in several heads, with slightly different answers.
  |  By Matt LeRay
A metric says latency increased. A log says a request failed. A trace identifies the slow dependency. An APM agent points to the method. Manual instrumentation explains the business operation. Traffic capture shows the exact request and response that triggered it. Each layer answers a question the previous layer could not. Each also introduces a new cost, blind spot, and failure mode.
  |  By Speedscale
This video provides a step-by-step demonstration of how to record an application's real-world traffic and generate an automated sandbox environment for debugging and testing.
  |  By Speedscale
We spend thousands of dollars "token maxing" and running endless debugging cycles just to walk our LLMs through a problem. But is the AI actually failing, or are we just withholding the right environment? Giving your AI assistant its own sandbox to test hypotheses might just be the missing link in your development workflow.
  |  By Speedscale
Tired of out-of-control log expenses and manual logging discipline? Discover next-generation observability with Speedscale. By using eBPF to record full-fidelity data right off the wire, you can instantly run full-text searches, track down leaking PII, and securely map out credentials across HTTP, Postgres, gRPC, and more—all without writing a single log line. Learn more: speedscale.com.
  |  By Speedscale
Fighting AI Slop: The Missing Ingredient for Clean Code.
  |  By Speedscale
Forecast latency, throughput and headroom before every deploy.

Continuous Resiliency from Speedscale gives you the power of a virtual SRE-bot working inside your automated software release pipeline. Forecast the real-world conditions of every build, and know you’ll hit your SLO’s before you go to production.

Feed Speedscale traffic (or let us listen) and we’ll turn it into traffic snapshots and corresponding mock containers. Insert your own service container in between for a robust sanity check every time you commit. Understand latency, throughput, headroom, and errors -- before you release! The best part? You didn’t have to write any scripts or talk to anyone!

Automated Traffic Replay for Every Stakeholder:

  • DevOps / SRE Pros: Understand if your app will break or burn up your error budget before you release.
  • Engineering Leads: Let Speedscale use traffic to autogenerate tests and mocks. Introduce Chaos testing and fuzzing.
  • Application Executives: Understand regression/performance, increase uptime and velocity with automation.

Before you go to production, run the projection.