Blog

September 14, 2025
5 min read

The Integrity Chain: Guardians of Distributed Systems

Distributed systems cannot escape their fundamental limits:

Communication across nodes, services or geographies is never instantaneous
Network latency, partitions and the speed of light guarantee that eventual consistency is the default state

Yet systems must still process requests, enforce business rules and maintain integrity. To cope with these realities, I use a set of recurring patterns in Scalable Modeling that form the foundation for growth-ready scalable systems.

I call them the Three Guardians, and together they form the Integrity Chain:

Gatekeeper: the front-line validator that filters commands across consistency boundaries before they reach execution
Command Handler: the authoritative validator and executor of state changes inside a single consistency boundary
Event Handler: the asynchronous reactor that makes sure all parts of the system catch up after domain facts are published

The chain works as follows:

A command is checked by a Gatekeeper across consistency boundaries
If accepted, it is passed to a Command Handler, which validates it synchronously within its own consistency boundary and either executes it or rejects it
The Command Handler produces events as facts when state changes occur
Event Handlers consume these events and drive reactions across the system

Integrity Chain Overview

Note

In addition to the Three Guardians, Scalable Modeling also describes the Query Handler and the Data Processor. A Query Handler serves read requests by consulting a single view. A Data Processor, on the other hand, combines data from multiple sources or views to maintain derived models. These roles are important but relatively straightforward, so I have not explored them in detail in this post.

Example: Ordering and Archiving a Product

Consider a system where customers can order products, but products can also be archived (removed from sale):

A customer issues a command to Order Product X
The Gatekeeper checks that Product X exists and forwards the command
The Command Handler for the order validates the customer account and records the order if valid
The Event Handler later reacts to the OrderPlaced event and updates projections

At the same time, an administrator issues a command to Archive Product X:

The Gatekeeper for product archiving checks for open orders, but due to distribution it may see stale state and allow the archive request to pass
The Command Handler archives Product X inside its consistency boundary and emits ProductArchived
The Event Handler consumes this event, queries for open orders of Product X and issues compensating actions such as canceling those orders and refunding customers. It waits long enough for the open orders view to catch up so it reflects all orders for Product X after the product was archived.

This demonstrates how the Integrity Chain works:

Gatekeeper prevents many invalid commands but cannot guarantee global freshness
Command Handler enforces invariants locally and emits authoritative events
Event Handler detects cross-boundary conflicts and reconciles asynchronously

Ordering vs Archiving Conflict

Compensating Actions as First-Class Features

In distributed systems compensating actions are not merely technical cleanups. They are first-class business features and just as important as the original actions that triggered the need for correction.

It is rarely enough to simply delete or undo an operation. In complex domains such as logistics or finance values and states are constantly changing. An order may affect billing, a shipment may alter routing, an invoice may trigger tax reports. If a mistake occurs after one of these actions a dedicated correction process must be triggered. These processes must be explicitly modeled as part of the business domain.

Examples include:

Billing and payments: A correction flow issues refunds or credit notes rather than silently rolling back the invoice
Logistics: A compensating flow reroutes trucks, updates manifests or notifies partners
Regulation: A correction flow files a new report rather than erasing the old one

These compensation and correction flows handle the inevitable gaps between feasibility checks and actual execution in distributed systems. They ensure not only technical consistency but also real-world accountability especially in domains where financial or regulatory stakeholders are involved. Ignoring them leads to messy downstream edits and fragile workarounds.

In the Integrity Chain Event Handlers are the natural home for modeling such flows since they react asynchronously to established facts and can drive corrective action across boundaries.

Tradeoffs in Handling the Archive-Order Race

When a product is archived at the same time as new orders are placed, the system must reconcile the conflict. There are several ways to do this, each with different tradeoffs:

Rely on views and compensate later: Archive first, then let an Event Handler query an eventually consistent view of open orders. Once the view has caught up, compensating actions cancel or refund invalid orders. This keeps product and order flows loosely coupled, but reconciliation is delayed and customers may briefly see invalid orders.

Delayed compensation through Event Handler

Compensate immediately: When a product is archived, there is a short window where new orders may still slip in. If an order attempts to register to a product that has already been archived, the product rejects the registration. When the Event Handler sees this rejection, it issues a compensating action such as canceling the order.

Immediate compensation during archival

No single option is universally correct. The choice depends on the domain:

Finance may favor immediate compensation to protect against invalid transactions
E-commerce may accept delayed reconciliation for looser coupling and higher throughput
Logistics may mix strategies, blocking for some operations while reconciling asynchronously for others

The key is to recognize that compensation, consistency and coupling are all levers. Each domain must strike its own balance.

Why This Matters

In a globally distributed system, models such as products, customers and orders exist in multiple places. Their read copies may lag behind one another. The Integrity Chain embraces this reality:

Distributed Models

Prevention at the edge with Gatekeepers
Enforcement within consistency boundaries via Command Handlers
Reconciliation across the system through Event Handlers

The result is integrity without requiring impossible global synchrony.

August 8, 2025
6 min read

Software’s Real Cost Is Building Wrong Features

Every year companies spend millions building software that looks right in a demo but collects dust in production. The issue is not lazy engineers or indecisive stakeholders. It is a gap in understanding between the people who know the problem and the people who write the code.

This perspective draws on both global studies and recent Nordic research to show why teams ship features nobody uses and how to stop it.

The result: bloated backlogs, features that miss the mark and a steady drain on budget. An often-cited figure from an early 2000s Standish Group report suggests that around 64% of features in delivered software saw little or no use¹. More recent large-scale product analytics paint an even starker picture: Pendo’s 2019 Feature Adoption Report, based on usage data from over 600 companies, found that about 80% of features were rarely or never used². While the exact percentage varies between studies, the underlying pattern that many features go unused remains widely observed. If you are funding those features you are paying for waste.

The good news is that many of the most expensive misunderstandings can be addressed before the first line of code is written. Coding will still raise new questions but with a shared understanding in place those iterations are faster, cheaper and far less painful.

Why This Problem Hurts the Bottom Line

Misunderstood requirements create a chain reaction of cost overruns and missed value.

NASA’s own analysis shows that fixing a requirements defect during design can be 3× to 8× more expensive than catching it in the requirements phase. In integration and test phases the cost jumps to 21× to 78×, and in extreme cases (when found in production) it can exceed 1 500×⁵
Research by Barry Boehm and others has shown that 30–50% of development effort is often spent on avoidable rework, much of it caused by misunderstood or incomplete requirements⁶
In a Nordic study, engineers spent over 16 hours per week in meetings, much of it dealing with misunderstandings that could have been avoided through earlier shared understanding³
A Swedish multi-company study found that weak alignment between requirements work and testing leads to rework, quality defects and delivery delays even when stakeholders believe they have a shared understanding⁴

These are not just statistics. They are your budget leaking out of the project one misunderstanding at a time.

How the Gap Opens

When a project jumps from Purpose & Problem (Why) straight to Implementation (How), there is no shared bridge between the two.

Domain experts focus on why the software is needed and who needs it. Engineers focus on how it will be implemented.

Without a shared middle ground, intent is often lost. The result is software that works technically but fails to solve the real problem, leading to expensive rework and wasted features.

In the Nordic study, many of those 16 hours per week spent in meetings were used to realign teams after these gaps emerged. Instead of moving work forward, time was spent repairing shared understanding that should have been built before implementation began³.

The Swedish study showed a similar pattern. Requirements and testing teams often worked from different assumptions, lacked traceability between specifications and verification steps and coordinated too late. This fragmented approach slowed delivery and created duplicate effort⁴.

Closing the Gap

Quote

"There is nothing so useless as doing efficiently that which should not be done at all."

Peter F. Drucker

The missing link is the Conceptual Model (What).

This is the shared understanding of how the solution will address the purpose and problem. It is built collaboratively between domain experts, engineers, architects and designers before implementation begins.

By making the Conceptual Model explicit, teams ensure:

The purpose is understood
The scope is clear
The transition from concept to implementation is smooth

Why Early Alignment Saves Money

Iteration cost is low when teams adjust at the Conceptual Model stage. Feedback at this level is fast, inexpensive and easy to act on.

Once a project moves into Implementation, changes become far more expensive and slow to deliver.

Early alignment does not remove the need for iteration. It ensures that when new questions or discoveries come up during coding they can be addressed faster, with fewer side effects and at a much lower cost.

Validating the Purpose & Problem and the Conceptual Model early gives teams:

Cheaper changes
Faster learning cycles
Fewer costly surprises in delivery

Tools That Close the Gap

Domain-Driven Design (DDD)

DDD is a way to embed the business language directly into the design and code.

Ubiquitous Language locks meaning to agreed terms
Bounded Contexts prevent rules from bleeding across domains
Collaborative modeling ensures experts stay involved until the intent is clear

These practices also address one of the Swedish study’s key findings that closer, earlier collaboration between requirements and testing roles improves alignment and reduces wasted effort⁴.

Event Storming

A fast, visual workshop where engineers, architects, designers and experts map business events together.

Reveals hidden complexity in hours
Surfaces misunderstandings before coding begins
Creates shared ownership of the solution

The Payoff

Addressing the expert–engineer gap is not a nice-to-have. It is a cost-control measure.

Teams that invest in collaborative domain exploration:

Deliver features that get used
Reduce rework and budget overruns
Build systems that align with the real business

Final Word

Quote

"It's developer (mis)understanding that's released in production, not the experts' knowledge."

Alberto Brandolini

Software is translation. Every mistranslation costs money. DDD and Event Storming will not remove all project risk but they make misunderstandings expensive to ignore and cheap to fix.

If you are tired of paying for features nobody uses bring your experts and engineers into the same room before the first sprint. The savings start before the code does.

And remember: increasing developers’ domain understanding is necessary, but not sufficient. Sustainable success also depends on an iterative, user-centric approach where real usage feedback shapes the product continuously. Without that loop, even well-understood domains risk delivering features that miss the mark.

Standish Group (2002). CHAOS Report figure cited by Mike Cohn in LinkedIn article. This is an older data point used here as an illustration of the problem, not a current benchmark. ↩
Pendo (2019). Feature Adoption Report. Based on usage data from 615 companies, showing that about 80% of features are rarely or never used. Report PDF ↩
Stray, V. G., & Moe, N. B. (2020). Understanding Coordination in Global Software Engineering: A Mixed-Methods Study on the Use of Meetings and Slack. arXiv:2007.02328. PDF ↩↩
Bjarnason, E., Rasmusson, A., Unterkalmsteiner, M., Engström, E., & Gorschek, T. (2023). Challenges and Practices in Aligning Requirements with Verification and Validation: A Case Study of Six Companies. arXiv:2307.12489. PDF ↩↩↩
NASA (2010). Cost of Fixing Defects. Based on analysis of NASA software projects and multiple independent studies. Relative cost multipliers: 3×–8× in design, 7×–16× in build, 21×–78× in integration/test, and up to 1 500× in extreme post-deployment cases. PDF ↩
Boehm, B., & Papaccio, P. N. (1988). Understanding and Controlling Software Costs. IEEE Transactions on Software Engineering, 14(10), 1462–1477. doi:10.1109/32.6191 ↩

January 7, 2025
3 min read

From CRUD to Real-Time Analytics: Why Business Events Matter

Many operational systems (e.g. SaaS platforms and transactional systems) are built around CRUD operations and batch processing, but these approaches often fall short when businesses demand real-time insights.

A common intermediate step is Change Data Capture (CDC), which reflects changes at the database level in near-real time. However, CDC primarily captures raw data changes (e.g., inserts, updates, deletes) and doesn’t provide the context of what those changes mean in the real world. For example, CDC might log an update to a database row, but it doesn’t tell you that the event was actually an “Order Shipped”.

This is where business events come in. Business events are high-quality, domain-specific data: they describe meaningful occurrences like “Payment Processed” or “Customer Registered”. Unlike raw data changes, these events provide the context and structure needed to power real-time analytics and unlock the full potential of AI and machine learning. These events, often called domain events, form the backbone of event-driven systems.

Business/Domain Events in Practice

For reliably generating, publishing and consuming these domain events, patterns like the transactional outbox, subscription, and event sourcing play crucial roles:

Transactional outbox ensures that events are published consistently as part of the same database transaction that writes the underlying data, providing robustness even during failures.
Subscription pattern enables systems to listen for and react to specific events in near real-time, enabling event-driven workflows and processes.
Event sourcing goes further by making events the source of truth, storing all changes as an immutable sequence of events — empowering systems with a full audit history and the ability to reprocess or replay events at any time.

By adopting event-driven architectures and producing event streams, organizations can move beyond traditional paradigms. This approach enables:

Timely, actionable insights through real-time analytics. A foundation of rich, contextual data for AI and ML applications. Real-time notifications to trigger actions and inform stakeholders instantly. Better system design with domain-driven thinking at the core.

Conclusion: Core domains in modern scalable operational systems should embrace business/domain events over Change Data Capture (CDC). Domain-driven design and event-driven architecture pave the way for creating future-proof systems.

Scalable modeling as an event-centric approach provides a strong foundation for building long-lasting operational systems that deliver low latency and enable real-time analytics at any scale.

October 1, 2024
3 min read

SW Skills Underrated: Being Easy to Work With

What are some ways you’ve worked on becoming easy to work with? Asking for a friend… 👀

Let’s face it, some folks are just easy to work with, and it doesn’t always have anything to do with having the same goals or interests. From my experience, those who are easy to work with tend also to be successful in their careers. I believe being easy to work with is essential for success in any field, but it’s especially important in software engineering.

Why do I believe being easy to work with is especially important in software engineering?

Software is ultimately a model, a conceptual solution that, while invisible, solves real-world challenges. In software engineering, two things matter: the conceptual solution (designing the what) and how it is implemented (developing the how). End users usually care much less about the technical details than about how well the solution solves their problems, so conceptual solution needs to be rock solid. This is where things like Domain-Driven Design, Systems Thinking & Residuality Theory for example, can help.

Great software design happens through iterative teamwork where people from diverse backgrounds come together to tackle complex, abstract problems and shape this conceptual solution. The challenge isn’t just technical: because of our diverse mental models, people often speak the same language but mean different things. For software success, teams need to create a shared language through conversations that ensure alignment on key terms and ideas. One efficient technique to use here is EventStorming, a collaborative workshop method that helps teams map out complex processes.

This is where being easy to work with becomes critical. In an industry that often focuses on technical skills, the ability to foster productive, clear, and empathetic communication can be the 𝘳𝘦𝘢𝘭 difference between project success and failure. When you’re easy to work with, you make those conversations productive and create the shared understanding that drives success.

Here’s how I’ve tried to practice it:

Be clear and concise in communication to reduce misunderstandings (you get extra points for adding an agenda to meeting invitations).
Listen actively to understand others’ perspectives before jumping to conclusions (you know, instead of nodding while thinking about the next thing you want to say).
Stay open to feedback and adjust your approach when needed. Maintain a positive attitude, especially when facing challenges or disagreements (a smile during tough moments goes a long way).
Foster trust and empathy by showing genuine care for others’ perspectives and creating a supportive environment. Trust grows when people feel understood and valued, making collaboration smoother and more effective.

I truly believe that being easy to work with is the most underrated skill in the tech-focused software industry, and it just might be the key to your success. So, how do you ensure you’re easy to work with on your team?

March 12, 2024
5 min read

Beyond the Stack: How Organizational Structure Shapes Software Architecture

The work of a software architect is highly technical, and that is also how the role is commonly perceived. However, what often gets overlooked is the sociological aspect of this discipline. No matter how well the architecture is described, the success of putting it into practice ultimately depends on the collaboration and understanding among the people involved.

It’s common sense that if we restrict communication channels between teams building a solution, the resulting software architecture is likely to mirror this fragmentation.

Restricting communication can lead to a push towards siloing

Let’s also consider a bit more complicated scenario where different teams are responsible for various layers of the software stack. Layers could include, for example, frontend, backend and database. Each team, focusing on its specialized layer, may unintentionally create a system that lacks cohesion and seamless integration. Technological boundaries are just one example among many other boundaries like business domains, availability, or location.

To spice up the scenario, let’s add a second boundary in addition to technology: location. Physical separation, be it due to geographical distances or departmental divisions, can also leave its imprint on the architecture. If your development group is split across two different locations, with local subgroups communicating face-to-face daily, expect the architecture to reflect this physical dispersion.

When speaking the same (technological or other) ubiquitous language, the communication is much more efficient. Same thing when communicating face to face - it just is more effortless than communication via technical communication channels.

Communication follows the path of least resistance

Communication tends to naturally follow the path of least resistance, favoring the most effortless channels.

In the example organization shown above, it makes sense that the resulting system structure would naturally start to resemble the picture below due to the least resistance principle.

Underlying forces at play

As we consider reshaping the architecture, it’s crucial to acknowledge and understand the strong underlying forces that may attempt to revert it to its original form.

Underlying forces trying to fragment the architectural attempts

To address these challenges, we need to make thoughtful changes in how our organization communicates. This involves adjusting the way teams interact, tackling the issues also raised by Conway’s Law, as outlined by Melvin Conway in his 1968 article “How Do Committees Invent?”.

Conway's Law

Melvin E. Conway, How Do Committees Invent?

Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.

Conway’s Law challenges the traditional notion of architecture as a purely technical discipline. It emphasizes the social fabric within an organization, asserting that the patterns of communication, collaboration, and cooperation are integral to shaping the software architecture.

Conclusions from the article “How Do Committees Invent?” by Melvin Conway

While Conway’s Law is a conceptual framework rather than a law in the scientific sense, there is empirical evidence and anecdotal support that suggests its validity in various contexts. Several case studies and observations in the field of software development and organizational behavior provide evidence that aligns with Conway’s Law.

Understanding Conway’s Law provides a roadmap for intentional and positive architectural design and raises the question: Should you design your organization with the awareness that how your teams are structured will inevitably shape the software they create? Answer is, yes.

Guiding force for positive change

The exciting prospect lies in transforming Conway’s Law from a constraint into an asset through the implementation of the Inverse Conway Maneuver. This proactive strategy empowers teams to deliberately shape communication and collaboration structures within an organization, leading to a positive impact on and enhancement of the resulting software architecture. Instead of simply adapting to existing communication patterns, teams take charge by intentionally designing the organizational structure to align with their desired outcomes in software development.

Implementing the following principles can help ensure that your organization’s structure fosters a conducive environment for positive software development outcomes:

Cross-functional Collaboration: Encourage collaboration across different disciplines. Break down the barriers that lead to siloed thinking and foster a culture of shared goals.
Flexible Team Structures: Design teams with flexibility in mind. Adapt to the evolving needs of the project rather than sticking rigidly to predefined roles.
Open Communication Channels: Emphasize open and transparent communication. This not only includes formal channels but also informal ones, encouraging spontaneous exchanges of ideas and information.
Shared Vision and Goals: Ensure that teams share a common vision and goals. Aligning everyone towards a unified objective helps create a cohesive software architecture that reflects the organization’s overarching mission.

By intentionally incorporating these principles into your organizational structure, you leverage Conway’s Law as a guiding force for positive change. Your software architecture becomes a reflection of a well-aligned and collaborative team, resulting in more robust and effective solutions.

Conclusions

As communication follows the path of least resistance and as building a larger scale solution is all about communication it is natural that the architecture starts to resemble the natural communication flows within the organization building it. We can leverage this by encouraging cross-functional collaboration, making team structures flexible, opening communication restrictions and putting effort into sharing vision and goals. The software architecture and communication structures go hand in hand and therefore software architecture is sociotechnical discipline.