Coding Blocks

Our journey into the world of DevOps continues with The DevOps Handbook as Michael doesn’t take enough tangents, Joe regrets automating the build, err, wait never regrets (sorry), and ducks really like Allen.

If you’re reading these show notes via your podcast player, you can find this episode’s full show notes at https://www.codingblocks.net/episode137, where you can be a part of the conversation.

Sponsors

  • Datadog – Sign up today for a free 14 day trial and get a free Datadog t-shirt after your first dashboard.
  • Secure Code Warrior – Start gamifying your organization’s security posture today, score 5,000 points, and get a free Secure Code Warrior t-shirt.

Survey Says

What is your meeting limit?

Take the survey at: https://www.codingblocks.net/episode137.

News

  • Thank you to everyone that left us a new review!
    • iTunes: justsomedudewritingareview, stupub, andrew.diamond, scipiomarcellus
    • Stitcher: Bicycle Repairman, BrunoLC
  • Using Bazel to build and test software of any size, quickly and reliably. (bazel.build)
  • Reflections on Trusting Trust (cs.cmu.edu)
  • Build your own Linux distro! (linuxfromscratch.org)
  • That System76 Oryx Pro keyboard though? (System76)

Fast, Reliable. Pick Two

Continuously Build, Test, and Integrate our Code and Environments

  • Build and test processes run constantly, independent of coding.
  • This ensures that we understand and codify all dependencies.
  • This ensures repeatable deployments and configuration management.
  • Once changes make it into source control, the packages and binaries are created only ONCE. Those same packages are used throughout the rest of the pipeline to ensure all environments are receiving the same bits.
  • What does this mean for our team culture?
    • You need to maintain reliable automated tests that truly validate deploy-ability.
    • You need to stop the “production pipeline” when validation fails, i.e. pull the andon cord.
    • You need to work in small, short lived batches that are based on trunk. No long-lived feature branches.
      • Short, fast feedback loops are necessary; builds on every commit.

Integrate Performance Testing into the Test Suite

  • Should likely build the performance testing environment at the beginning of the project so that it can be used throughout.
  • Logging results on each run is also important. If a set of changes shows a drastic difference from the previous run, then it should be investigated.

Enable and Practice Continuous Integration

  • Small batch and andon cord style development practices optimize for team productivity.
  • Long lived feature branches optimize for individual productivity. But:
    • They require painful integration periods, such as complex merges, which is “invisible work”.
    • They can complicate pipelines.
    • The integration complexity scales exponentially with the number of feature branches in play.
    • They can make adding new features, teams, and individuals to a team really difficult.
  • Trunk based development has some major benefits such as:
    • Merging more often means finding problems sooner.
    • It moves us closer to “single piece flow”, such as single envelope at a time story, like one big assembly line.

Automate and Enable Low-Risk Releases

  • Small batch changes are inherently less risky.
  • The time to fix is strongly correlated with the time to remediate, i.e. the mean time to find (MTF) and the mean time to remediate (MTR).
  • Automation needs to include operational changes, such as restarting services, that need to happen as well.
  • Enable “self-service” deployments. Teams and individuals need to be able to dynamically spin up reliable environments.

Decouple Deployments from Releases

  • Releases are marketing driven and refer to when features are made available to customers.
  • Feature flags can be used to toggle the release of functionality independent of their deployments.
  • Feature flags enable roll back, graceful degradation, graceful release, and resilience.

Architect for Low-Risk Releases

  • Don’t start over! You make a lot of the same mistakes, and new ones, and ultimately end up at the same place. Instead, fix forward!
  • Use the strangler pattern instead to push the good stuff in and push the bad stuff out, like how strangler vines grow to cover and subsume a fig tree.
  • Decouple your code and architecture.
  • Use good, strong versioned APIs, and dependency management to help get there.

Resources We Like

  • The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations (Amazon)
  • The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win (Amazon)
  • The Unicorn Project: A Novel about Developers, Digital Disruption, and Thriving in the Age of Data (Amazon)
  • Andon (manufacturing) (Wikipedia)
  • LEGO Nintendo Entertainment System (lego.com)
  • Comparing Git Workflows (episode 90)
  • StranglerFigApplication (MartinFowler.com)

Tip of the Week

  • Be sure to check out the gaming channel in Slack to find and connect with some great people for your next game.
    • Also, look up in the Slack channel. There may be cool information in the channel’s description.
  • Within JetBrains IDEs, such as Datagrip and IntelliJ, press CTRL+SHIFT+V to see and choose from the list of items recently copied to the clipboard.
  • Tired of working from home? Work from a park!
  • Save your sanity by customizing your Macbook Pro Touchbar in Chrome by going to View -> Customize Touchbar.
Direct download: coding-blocks-episode-137.mp3
Category:Software Development -- posted at: 8:01pm EDT

We begin our journey into the repeatable world of DevOps by taking cues from The DevOps Handbook, while Allen loves all things propane, Joe debuts his “singing” career with his new music video, and Michael did a very bad, awful thing.

These show notes can be found at https://www.codingblocks.net/episode136 for those reading this via their podcast player.

Sponsors

  • Datadog.com/codingblocks – Sign up today for a free 14 day trial and get a free Datadog t-shirt after your first dashboard.

Survey Says

When you do listen to music, how do you do it?

Take the survey at: https://www.codingblocks.net/episode136.

News

  • We’re very thankful for all of the new reviews!
    • iTunes: galTheJewishHammer, Ragnaroekk, Marcel7473826, rkosko31
    • Stitcher: Bicycle Repairman, BrunoLC
  • Joe serenades us in song! (YouTube)
  • Congratulations Allen on his renewed MVP status!

What is The DevOps Handbook?

  • It’s a collection of arguments and high level guidance for understanding the spirit of DevOps.
  • It’s light on specifics and heavy on culture. The tools aren’t the problem here, the people need to change.
  • It’s also a book about scaling features, teams, people, and environments.

The First Way: The Principles of Flow

The Deployment Pipeline is the Foundation

Continuous delivery:

  • Reduces the risk associated with deploying and releasing changes.
  • Allows for an automated deployment pipeline.
  • Allows for automated tests.

Environments on Demand

  • Always use production like environments at every stage of the stream.
  • Environments must be created in an automated fashion.
  • Should have all scripts and configurations stored in source control.
  • Should require no intervention from operations.

The reality though …

  • Often times the first time an application is tested in a production like environment, is in production.
  • Many times test and development environments are not configured the same.

Ideally though …

  • Developers should be running their code in production like environments from the very beginning, on their own workstations.
    • This provides an early and constant feedback cycle.
  • Rather than creating wiki pages on how to set things up, the configurations and scripts necessary are committed to source control. This can include any of all of the following:
    • Copying virtualized environments.
    • Building automated environments on bare metal.
    • Using infrastructure as code, i.e. Puppet, Chef, Ansible, Salt, CFEngine, etc.
    • Using automated OS configuration tools.
    • Creating environments from virtual images or containers.
    • Creating new environments in public clouds.

All of this allows entire systems to be spun up quickly making this …

  • A win for operations as they don’t have to constantly battle configuration problems.
  • A win for developers because they can find and fix things very early in the development process that benefits all environments.

“When developers put all their application source files and configurations in version control, it becomes the single repository of truth that contains the precise intended state of the system.”

The DevOps Handbook

Check Everything into One Spot, that Everybody has Access to

Here are the types of things that should be stored in source control:

  • All application code and its dependencies (e.g. libraries, static content, etc.)
  • Scripts for creating databases, lookup data, etc.
  • Environment creation tools and artifacts (VMWare, AMI images, Puppet or Chef recipes).
  • Files used to create containers (Docker files, Rocket definition files, etc.)
  • All automated tests and manual scripts.
  • Scripts for code packaging, deployments, database migrations, and environment provisioning.
  • Additional artifacts such as documentation, deployment procedures, and release notes.
  • Cloud configuration files, such as AWS CloudFormation templates, Azure ARM templates, Terraform scripts, etc.)
  • All scripts or configurations for infrastructure supporting services for things like services buses, firewalls, etc.

Make Infrastructure Easier to Rebuild than to Repair

  • Treat servers like cattle instead of pets, meaning, rather than care for and fix them when they’re broken, instead delete and recreate them.
  • This has the side effect of keeping your architecture fluid.
  • Some have adopted immutable infrastructure where manual changes to environments are not allowed. Instead, changes are in source control which removes variance among environments.

The Definition of Done

  • “Done” means your changeset running in a production-like environment.
  • This ensures that developers are involved in getting code to production and bring operations closer to the code.

Enable Fast and Reliable Automated Testing

  • Automated tests let you move faster, with more confidence, and shortens feedback cycles for catching and fixing problems earlier.
  • Automated testing allowed the Google Web Server team to go from one of the least productive, to most productive group in the company.

Resources We Like

  • The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations (Amazon)
  • The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win (Amazon)
  • The Unicorn Project: A Novel about Developers, Digital Disruption, and Thriving in the Age of Data (Amazon)
  • Kubernetes Failure Stories (k8s.af)

Tip of the Week

  • Press SHIFT twice to search everywhere in your IntelliJ project: Search for a target by name (Search everywhere) (JetBrains)
  • Use ALT+F1 in Datagrip to see available options for a schema object such as navigating to it in the left pane.
Direct download: coding-blocks-episode-136.mp3
Category:Software Development -- posted at: 8:01pm EDT

1