Skip to content
September 9, 20266 min readBy Dzaki Amri Zaidaan

Lotus Notes and the Dangers of Starting from Scratch: A Technical Postmortem

Lotus Notes dominated enterprise collaboration for decades, yet its decline offers a masterclass in the risks of greenfield rewrites. This article dissects the architectural decisions that made Notes powerful yet brittle, and why incremental evolution beats starting from scratch.

#Architecture
Lotus Notes and the Dangers of Starting from Scratch: A Technical Postmortem - Deep learning neural network graphic

1. The Problem & Industry Shift

In the 1990s, Lotus Notes was the undisputed king of enterprise collaboration. It combined email, document management, workflow automation, and application development into a single platform. By 1995, it had over 10 million users [1]. Yet, by the 2010s, Notes was widely derided as legacy bloat, and IBM eventually sold the product to HCL Technologies in 2018 [2].

Why did Notes fall from grace? The common narrative blames poor UI or IBM's mismanagement. But a deeper technical reason is the dangers of starting from scratch. When IBM acquired Lotus in 1995, they faced a choice: evolve Notes incrementally or rebuild it on modern architecture. They chose the latter, leading to a series of failed rewrites (e.g., the 'NextGen' project) that drained resources and delayed innovation [3].

This pattern is not unique to Notes. Many engineering teams, facing a messy legacy codebase, dream of a clean rewrite. Yet, as Joel Spolsky famously argued, 'The single worst strategic mistake that any software company can make is to rewrite the code from scratch' [4]. The Notes story is a case study in why that advice holds, even for a product with a decade of market dominance.

2. Architecture & Core Mechanics

To understand the rewrite trap, we must first appreciate what made Notes work. Its architecture was revolutionary for its time:

  • Document-oriented database: Unlike relational databases, Notes stored data as semi-structured documents (NSF files). This allowed flexible schemas, ideal for collaborative workflows.
  • Replication engine: Notes replicated data across servers and clients, enabling offline access and eventual consistency—long before NoSQL made this trendy.
  • Application development environment: Notes included a built-in IDE (LotusScript, @Formula) that allowed non-programmers to build custom applications, such as approval workflows and discussion forums.

Here's a simplified data flow of a Notes application:

[Client (Notes Client)] 
        |
        | (RPC over TCP/IP)
        v
[Domino Server] ---> [NSF Database]
        |
        | (Replication)
        v
[Other Domino Servers]

This architecture was powerful but had deep coupling. The client, server, database, and scripting language were tightly integrated. Any attempt to modernize one component without the others risked breaking the entire ecosystem.

3. Production Code Example: The Rewrite Trap in Practice

Let's illustrate the danger with a concrete example. Suppose you have a legacy Notes application that tracks customer complaints. The code uses @Formula, a declarative language. Here's a typical piece of logic:

' Legacy Notes code (LotusScript)
Sub Queryopen(Source As Notesuidocument, Mode As Integer, Isnewdoc As Variant, Continue As Variant)
    Dim session As New NotesSession
    Dim db As NotesDatabase
    Set db = session.CurrentDatabase
    Dim doc As NotesDocument
    Set doc = Source.Document
    
    ' Check if customer is VIP
    If doc.CustomerType(0) = "VIP" Then
        doc.Priority = "High"
    Else
        doc.Priority = "Normal"
    End If
    Continue = True
End Sub

Now, a team decides to rewrite this in a modern stack (e.g., React + Node.js + MongoDB). They start from scratch, aiming to replicate all features. But they quickly discover hidden requirements:

  • The original application had 500 forms and 200 agents, many undocumented.
  • Business rules were embedded in UI events, not in a central logic layer.
  • Replication conflicts were handled by Notes' built-in conflict detection, which the new system must reimplement.

After 18 months, the rewrite is incomplete, and the team has spent 10x the original development cost. Meanwhile, the legacy system still runs the business, but no one wants to touch it.

Key engineering decision: Instead of a full rewrite, a better approach is strangler pattern—incrementally replace parts of the system while keeping the whole operational. For example, you could expose Notes data via REST APIs and build new frontends, gradually migrating features.

4. Performance, Cost & Trade-offs

The rewrite trap is not just about lost time; it has measurable technical costs:

  • Performance: Notes' replication was optimized for low-bandwidth connections. A modern rewrite using synchronous HTTP calls would perform worse in offline scenarios. Benchmarks from the late 90s showed Notes could handle thousands of concurrent users on modest hardware, but a modern stack might require more resources.
  • Cost: IBM's NextGen rewrite consumed millions of dollars and years of effort, yet never shipped. The opportunity cost was enormous—Notes' market share eroded as competitors like SharePoint and Google Apps emerged.
  • Security: Notes had a mature security model with ACLs and encryption. A rewrite might introduce vulnerabilities if not carefully designed.

Trade-offs: A rewrite offers a chance to modernize architecture, but it sacrifices years of battle-tested behavior. The longer the legacy system has run, the more business logic it encodes, and the riskier a rewrite becomes.

5. Actionable Checklist / Summary

When considering a rewrite, follow these guidelines:

  1. Audit your legacy system: Document all features, integrations, and hidden business rules. Use code analysis tools and interviews with power users.
  2. Consider incremental evolution: Use the strangler pattern to gradually replace components. For Notes, this might mean building a new frontend that calls Domino REST APIs.
  3. Quantify the cost: Estimate the time and money for a rewrite, and compare it to the cost of maintaining the legacy system. Include the risk of losing institutional knowledge.
  4. Prototype risky features: If you must rewrite, build a proof-of-concept for the most complex features first. If they fail, you've saved yourself from a full-scale disaster.
  5. Plan for data migration: Data is often the most valuable asset. Ensure you have a robust migration strategy that preserves relationships and history.

6. References