In-progress design overview

Pull request

Table of contents

Problem

We need some vaguely consistent shared understanding of the language design as a whole. The goal is merely consistency in discussions, in-progress syntax, and general approach. It in no way needs reliably to reflect the end state, either desired or realized. Instead, it should evolve as each area matures and becomes concrete, while providing an overarching overview that connects the language together, and the consistent background that we all can refer back to during discussions.

Goals

This is intended to offer a reasonable starting point for:

  • Example code.
  • Conceptualizing Carbon at a high level.
  • Reasonable, but not necessarily final, approaches to features in README.md.
    • If any idea is obviously bad, we can clean it up here.

This proposal is not intended to achieve:

  • A whole language design.
    • This is way too much work for a single proposal; this is a skeletal framework only.
    • As we work on feature-specific designs, we may decide to use other approaches. That’s fine: we only need somewhere to start.
    • The summaries in README.md may be expected to change over time.
  • Feature-specific files aren’t intended to be well-written or comprehensive. They are a quick jot of prior thoughts.
    • We want to avoid getting stuck on language details that we should consider more carefully regardless. If you’re passionate about a feature, please feel free to start a new proposal for it.
    • Each and every aspect of the suggested overview should be subject to careful examination and justification before it becomes a settled plan of record.

Background

Many of the ideas here stem from discussions between several of the initial people working on Carbon over several years. That doesn’t make them good, but may give some context on where they came from. They are also heavily informed by the experience several of us have both working on the Clang C++ frontend and several C++ codebases including those of LLVM and Clang themselves.

Proposal

See the language design overview document.

Alternatives considered

Single-file design overview

We also considered putting the full design overview in one file, as in PR 22. This is versus the hierarchy proposed here.

Advantages:

  • All proposed changes are in one place.
  • Easier for people to skim rationale and considered changes.

Disadvantages:

  • Encourages more single-file designs.
    • A principle of the multi-file approach is that complex features may have subdirectories with their own README.md and files for sub-features, similar to the relationship between this design overview and features.
    • Single-file designs may be harder to evolve long-term, as the volume of information contained impedes reading.

No in-progress design overview

The primary alternative is to avoid even having a draft or in-progress design of this form until each constituent component is more thoroughly conceived and considered.

Advantages:

  • Avoids anchoring design on approaches that haven’t yet been fully explored.
    • Avoids getting stuck on discussing details where a proposal isn’t fleshed out.

Disadvantages:

  • The lack of an overview can lead to significant confusion and inconsistencies in discussion, hindering fleshing out details.
    • An overview offers basic shaping of the language as a whole, even as it evolves.

The compromise chosen is to have the in-progress design and simply work to resist both anchoring and distraction stemming from it. We want to get the benefits we can here while minimizing the cost.

No overview of designs

The overview will result in content duplication from individual designs. At the time of this proposal, this may be significant because individual designs are not fleshed out, and should thus duplication should be expected to reduce over time. However, it should be expected to remain as the duplication is fundamental to having an overview.

This duplication could be addressed by removing the overview. Instead, design/README.md could be restricted to listing existing designs, with no additional content.

The proposed approach assumes that the proposed overviews offer significant value for ramp-up.

Advantages:

  • Eliminates content duplication.
  • A simple index is easier to maintain long-term, with less to become stale.
    • It could be fully automated.

Disadvantages:

  • No quick way to get a high-level understanding.
    • The overview is the only step before “reading every design”.
    • For example, we summarize common control flow keywords, so that readers don’t need to identify which documents they come from and what exists.
  • Harder to show relationships between various features.
    • While examples can show how designs relate, it may not be as obvious from a simple link, even when reading the associated design.
    • For example, lexical conventions come up as references for three otherwise distinct sections. If we had a simple index of files, we should expect users to need to read individual designs to understand relationships.
    • For example, we explain in brief the relationships between categories of types.
    • There’s disagreement about whether the text of README.md offers any utility: comment thread

Rationale

This proposal provides some much-needed context for a lot of our discussions and deliberations about Carbon. Having a skeleton in place for the language design is an important step forward. We can’t make language design decisions in a vacuum – we need a “blurry outline” of the big picture before we can start filling in the details of any particular area. We need to establish a common frame of reference regarding the overall shape of the language, so that we can parallelize the in-depth design work.

That means that, by necessity, this proposal suggests lots of concrete design decisions that have not yet had sufficient analysis for us to affirm them. There is a shared understanding that we are not committing to any of those decisions, only to the broad picture painted by the combination of those decisions, and that all such decisions need to be revisited by another proposal before we consider them to be agreed on. There is a substantial risk of anchoring how we think about Carbon – we’ll just have to do our best to avoid taking this doc as a given when evaluating subsequent proposals. Those proposals must justify a direction that agrees with this doc as much as one that does not agree with it.

This doc sets the stage for increasingly incremental steps towards a complete design. Establishing a structure for the design is especially helpful as it will show how things do or do not connect across the language.

Adopting this largely work-in-progress overview in order to see the structure of things, while still needing to resolve the specifics in each area, will directly help reinforce our goal of language evolution over time. This will help us learn how to effectively iterate, and how to compensate and overcome the risks of anchoring, change aversion, and other challenges.