Sun, 29 September 2019
We discuss the second factor of Hasura’s 3factor app, Reliable Eventing, as Allen says he still _surfs_ the Internet (but really, does he?), it’s never too late for pizza according to Joe, and Michael wants to un-hear things.
This episode’s full show notes can be found at https://www.codingblocks.net/episode116, just in case you’re using your podcast player to read this.
Survey Says …
What's the first thing you do when picking up a new technology or stack?
- Thank you to everyone that left us a review:
- iTunes: !theBestCoder, guacamoly, Fishslider
- Stitcher: SpottieDog
- We have photographic evidence that we were in the same room with Beej from Complete Developer at Atlanta Code Camp.
The Second Factor – Reliable Eventing
- Don’t allow for mutable state. Get rid of in memory state manipulation in APIs.
- Persist everything in atomic events.
- The event system should have the following characteristics:
- Atomic – the entire operation must succeed and be isolated from other operations.
- Reliable – events should be delivered to subscribers at least once.
Comparing the 3factor app Eventing to Traditional Transactions
|Traditional application ||3factor application |
|Request is issued, data loaded from various storage areas, apply business logic, and finally commit the data to storage. ||Request is issued and all events are stored individually. |
|Avoid using async features because it’s difficult to rollback when there are problems. ||Due to the use of the event system, async operations are much easier to implement. |
|Have to implement custom recovery logic to rewind the business logic. ||Recovery logic isn’t required since the events are atomic. |
Benefits of an Immutable Event Log
- Primary benefit is simplicity when dealing with recovery. There’s no custom business logic because all the event data is available for replayability.
- Due to the nature of persisting the individual event data, you have a built in audit trail, without the need for additional logging.
- Replicating the application is as simple as taking the events and replaying the business logic on top of them.
Downsides of the Immutable Event Log
- Information isn’t (instantly) queryable, not taking into account snapshotting.
- CQRS (command query responsibility segregation) helps to answer this particular problem.
- Forcing event sourcing on every part of the system introduces significant complexity where it may not be needed.
- For evolving applications, changing business needs require changes to the event schema and this can become very complex and difficult to maintain.
- Upcasting: converting an event record on the fly to reflect a newer schema. Problem with this is you’ve now defeated the purpose of immutable events.
- Lazy upcasting is evolving the event records over time, but that means you’re now maintaining code that knows how to understand each version of the event records in the system, making it very difficult to maintain.
- Converting the entire set of data any time a schema needs to change. Keeps things in sync but at a potentially large cost of taking the hit to update everything.
- Considerations of event granularity, i.e. how many isolated events are too much and how few are not enough?
- Too many and there won’t be enough information attached to the event to be meaningful and useful.
- Too few and you take a major hit on serialization/deserialization and you run the risk of not having any domain value.
- So what’s the best granularity? Keep the events closely tied to the DDD intent.
- Fixing bugs in the system may be quite a bit more difficult than a simple update query in a database because events are supposed to be immutable.
Resources We Like
Tip of the Week
docker system to manage your Docker environment.
docker system df to see the the disk usage.
docker system prune to clean up your environment.
- How To View Clipboard History On Windows 10 (AddictiveTips.com)
docker-compose down -v to also remove the volumes when stopping your containers.
- Intel 660p M.2 2TB NVMe PCIe SSD (Amazon)
- Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems (Amazon)
Direct download: coding-blocks-episode-116.mp3
-- posted at: 10:53pm EDT
Mon, 16 September 2019
We begin to twitch as we review the first factor of Hasura’s 3factor app, Realtime GraphQL, while Allen gets distrac … SQUIRREL!, Michael might own some bell bottoms, and Joe is stuck with cobalt.
If you’re reading these notes via your podcast app, you can find this episode’s full show notes and join in on the conversation at https://www.codingblocks.net/episode115.
- Datadog.com/codingblocks – Sign up today for a free 14 day trial and get a free Datadog t-shirt after creating your first dashboard.
- O’Reilly Software Architecture Conference – Microservices, domain-driven design, and more. The O’Reilly Software Architecture Conference covers the skills and tools every software architect needs. Use the code
BLOCKS during registration to get 20% off of most passes.
- Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 20% off any course.
Survey Says …
Would you be interested in doing a Coding Blocks Fantasy Football League?
Take the survey here: https://www.codingblocks.net/episode115.
- To everyone that took a moment to leave us a review, thank you. We really appreciate it.
- iTunes: Zj, Who farted? Not me., Markus Johansson, this jus10, siftycat, Runs-With-Scissors
- Stitcher: wuddadid, unclescooter
- Zach Ingbretsen gives us a Vim tutorial: RAW Vim Workshop/Tutorial (YouTube)
3factor app and the First Factor
- The 3factor app is a modern architecture for full stack applications, described by the folks at Hasura.
- High feature velocity and scalability from the start:
- Real time GraphQL
- Reliable eventing
- Async serverless
- Kinda boils down to …
- Have an API gateway (for them, GraphQL).
- Store state in a (most likely distributed) store.
- Have services interact with state via an event system.
- Versus how did we used to do things using a REST API for each individual entity.
- Let’s be honest though. We probably created a single very specialized REST API for a particular page all in the name of performance. But it was only used for that page.
- Related technologies:
- Web Sockets
- Lambda / Kappa – Types Streaming architectures
- Event based architectures
Factor 1 – Realtime GraphQL
Use Realtime GraphQL as the Data API Layer
- Must be low-latency.
- Less than 100 ms is ideal.
- Must support subscriptions.
- Allows the application to consume information from the GraphQL API in real-time.
Some Comparisons to Typical Backend API Calls
|Traditional application ||3factor application |
|Uses REST calls. ||Uses GraphQL API. |
|May require multiple calls to retrieve all data (customer, order, order details) – OR a complex purpose built call that will return all three in one call. ||Uses GraphQL query to return data needed in a single call defined by the caller. |
|Uses something like Swagger to generate API documentation. ||GraphQL will auto-generate entire schema and related documents. |
|For realtime you’ll set up WebSocket based APIs. ||Use GraphQL’s native subscriptions. |
|Continuously poll backend for updates. ||Use GraphQL’s event based subscriptions to receive updates. |
Major Benefits of GraphQL
- Massively accelerates front-end development speed because developers can get the data they want without any need to build additional APIs.
- GraphQL APIs are strongly typed.
- Don’t need to maintain additional documenting tools. Using a UI like GraphiQL, you can explore data by writing queries with an Intellisense like auto-complete experience.
- Realtime built in natively.
- Prevents over-fetching. Sorta. To the client, yes. Not necessarily so though on the server side.
A Little More About GraphQL
- GraphQL is a Query Language for your API.
- It isn’t tied to any particular database or storage engine.
- It’s backed by your existing code and data.
- Queries are all about asking for specific fields on objects.
- The shape of your query will match the shape of the results.
- Queries allow for traversing relationships, so you can get all the data you need in a single request.
- Every field and nested object has its own set of arguments that can be passed.
- Many types are supported, including enums.
- GraphQL has the ability to alias fields to return multiple results of the same type but with different return names (think of aliasing tables in a database query).
- Fragments allow you to save a set of query fields to retrieve, allowing you to later reuse those fragments in simpler queries. This allows you to create complex queries with a much smaller syntax.
- There’s even the ability to use variables within the fragments for further queries requiring more flexibility.
- Three types of operations are supported: query, mutation, and subscription.
- Providing an operation name is not required, except for multi-operation documents, but is recommended to aid debugging and server side logging.
- Queries are typically dynamic by way of variables.
- Supported variable types are scalars, enums, and input object types.
- Input object types must map to server defined objects.
- Can be optional or required.
- Default values are supported.
- Using variables, you can shape the results of a query.
- Mutations allow for modifying data.
- Nesting objects allows you to return data after the mutation occurs,
- Mutations, unlike queries, run sequentially, meaning
mutation1 will finish before
- In contrast, queries run in parallel.
- Meta fields
- GraphQL also provides meta fields that you can use to inspect the schema that are part of the introspection system.
- These meta fields are preceded by a double underscore, like
- GraphQL schema language
- Objects are the building blocks of a schema.
- Fields are properties that are available on the object.
- Field return types are defined as well – scalar, enum or objects.
- Scalar types: Int, Float, String Boolean, ID (special use case), or User Defined – must specify serializer, deserializer and validator.
- Fields can also be defined as non-nullable with an exclamation after the type like
- This can be done on array types as well after the square brackets to indicate that an array will always be returned, with zero or more elements, like
- Each field can have zero or more arguments and those arguments can have default values.
- Lists are supported by using square brackets.
- GraphQL’s type system supports interfaces.
- Complex objects can also be passed as input types, however, they are defined as
input rather than
Resources We Like
Tip of the Week
- From Google’s Engineering Practices documentation: How to do a code review (GitHub.io).
- This is part of the larger Google Engineering Practices Documentation (GitHub.io).
CTRL+SHIFT+V to access Visual Studio’s Clipboard Ring.
- Take control of your tab usage in your browser with Workona.
- Theme your Chrome DevTools!
Direct download: coding-blocks-episode-115.mp3
-- posted at: 10:10pm EDT