The "Open Source" in CockroachDB

When I started contributing to CockroachDB, I was foremost excited to contribute to an open source project with a strong "free as in freedom" component, which makes CockroachDB agreeably FOSS in my eyes.

FOSS has a social purpose, one that supports giving others both the tools to progress forward and the keys to these tools, so that they can keep their full agency. At a more personal level, I expect to be changing jobs a couple more times before I retire, and keeping a public record of my contributions makes it easier for me to apply to new positions—me not being good with charisma and things, I need more evidence than average to advertise my skills.

Also, CockroachDB had just started! Being scientifically minded, I was curious to study the progression of this project in its early stages. Every open source project has a different narrative and its own dynamics, and I felt there was much more to learn by being close to the source than relying on third party retrospectives a few years later.

It has been more than two years since I started contributing to CockroachDB, and today I am sharing a few findings and observations.

CockroachDB as free and open source software (FOSS)

What makes CockroachDB free and open source software?

What makes CockroachDB an open source project?

Governance models and ownership

To start, in my experience, most FOSS comes in two flavors:

  • free and open source from the start, initially supported by volunteers or public funding (e.g. the GNU project, the Linux kernel, GCC, LLVM, Debian, PostgreSQL), then later, if successful, supported by private funding (e.g. LLVM, GCC, Linux, PostgreSQL).

  • initially proprietary and commercially funded, and freed later when the parent organization realizes they can't easily make money off it (e.g. ScyllaDB, OpenOffice/LibreOffice, RethinkDB, FoundationDB).

CockroachDB is none of those things. Instead, it is one of these few projects, alongside RedHat Linux, Ubuntu Linux, MySQL, Go and Rust, where the software is both created as FOSS upfront and funded commercially.

The reason why I start with this topic is that I am particularly interested in the social dynamics around ownership: how is the power shared?

Let's look at these three categories separately.

Classical and simple FOSS governance

In the first category, projects are created to either scratch the hacker's itch or for public good. The authors and, if applicable, the funding organization, give birth to an entity with the aim upfront to have its community self-organize and decide its governance. Most, if not all, have as founding principle that the project's governance is, nominally at least, open to the public via a documented and transparent process. A newcomer to the project can legitimately assume upfront that the project is partly theirs and that their influence on the project is guaranteed to grow in proportion to their contributions.

There is a pitfall where such a born-FOSS project becomes funded by a single for-profit organization, and evolves to serve primarily the interests of that organization instead of that of its community (LLVM/Clang is an example of this vis-a-vis Apple, more on that below) because the livelihood of most core contributors depends on that organization. Nevertheless, this is more an exception than the rule, and so far I have found that most healthy FOSS projects are careful to ensure that many of its core contributors are supported by different organizations.

In the second category, there is a business building new software more or less in private, in any case with a big entry sign in front of its door: "this product is ours, and you must give us money before we allow you to use it" (nowadays the money can be in the form of advertisement-receiving eyeballs, but the idea remains the same). There is initially no pretense of shared ownership, and the users are reminded regularly that they do not have any participatory rights.

Then, at some point, the business changes and abandons moral ownership over the software. This event can occur for a variety of reasons (bankruptcy, pivot, leadership change, whatever) which are not relevant here; instead, I want to focus on the observation that there is then a pivotal event where a community of users is handed the keys to a kingdom previously guarded, with a fundamental guarantee that the king has left the premises and is not coming back—usually via suitably irrevocable FOSS license, but it can also be organized via a copyright reassignment to a non-profit organization with an open charter. What happens then is that both existing users and some of the previous contributors gather and decide to create a new governance. In reaction to the previous closed ownership, the new governance is, again, nominally at least, open to the public via a documented and transparent process.

This was the story of e.g. StarOffice opening itself as OpenOffice and finally LibreOffice.

The result is organizationally similar to that described above, but the dynamics of the interaction between newcomers and a late-freed project are rather different. To understand why, consider this analogy: a born-freed FOSS project is like a social group building their shared home together on top of a vacant plot of land; a late-freed FOSS project, instead, is like a group moving in a giant mansion inherited from an old, quirky relative. There is outdated upholstery to get rid of; there are poorly maintained pipes to renovate at yet-undetermined costs, and, foremost, the overall architecture of the mansion was fundamentally built for someone else, not for the new inhabitants. Newcomers can legitimately feel that the project is partly "theirs" and that their influence can grow with their contributions, but there are days that bring reminders of the previous owner, much like ghosts of a haunted mansion.

Hybrid FOSS governance—spectrum of influence

The third category, where one can find RedHat and Ubuntu Linux, Go, Rust and now CockroachDB, is quite different. I'll call them "hybrid" FOSS projects.

In this case, there is a for-profit organization who bankrolls the development. Meanwhile, the source code is public and there is an open source license, even a FOSS one usually. Forks and other reuses of the FOSS code are permitted, but the name and/or the brand is trademarked, such that an individual or a third party organization cannot derive a new project from the same sources with the same name.

The governance model, in particular the process available to an external contributor to become acknowledged as part of a steering group, further depends on the structure of the business around the FOSS project. As for most things, there is a spectrum.

At one end of the spectrum, if the organization is merely building FOSS as an instrument to create a community on top of which it can sell services or other professional services, i.e. where the FOSS and/or related IP is not a leveraged asset of the for-profit, then the governance is usually public. Steering groups and/or "core teams" meet on public channels, the membership process is transparent, public and open to all, and the parent for-profit typically only acts in an advisory role. Ubuntu Linux was created in that way by Canonical, its parent company.

At the other end of the spectrum, the software is a core asset of the for-profit, which the for-profit expects to directly leverage for income (by selling deployment licenses or subscriptions, or by proposing it as a gateway drug to a more expensive, proprietary product). Out of business necessity, the steering is exclusively reserved to the for-profit, and the core team is closed to the public. The existence of the FOSS license is seen as a source of undesirable extra costs (support and communication with the community) and merely supported or tolerated for the free advertisement it generates. External contributions are accepted and integrated from external contributors under the condition that they relinquish their copyright to the parent organization, and there is no compensation in the form of participatory rights in the steering activities. Arguably, MySQL belongs more to this category.

And then there is a range of various FOSS projects between these two extremes. Where they belong is, again, a function of their governance model: how external contributions are welcomed, how external contributors can raise in the community to gain steering powers, and all in all how individual users and contributors can legitimately feel the software is "theirs".

The Rust project, currently bankrolled by Mozilla, is much more like Ubuntu Linux than it is like MySQL. All development occurs in public, including all the project steering. Any external contributor is given the opportunity to contribute to the project's steering via the RFC process—how much influence can be exercised this way being only a function of the contributor's own technical acumen and their reputation, as opposed to opaque decision making by the community. What distinguishes the governance in this case from the laissez-faire in Ubuntu's case, is that the FOSS project is steered primarily to support a core business in the parent organization: Firefox' Servo in the case of Rust, the IT services in the case of RedHat.

In contrast, one would find the Go project closer to MySQL on that spectrum than Rust. While development occurs in public, the steering process is opaque and driven solely by Google's own internal (non-public) uses of the Go language. While external contributions are welcome and the parent organization supports the community by funding events, etc., there is no process by which an external contributor can ascend in a steering position; changes to the core project (the language and the compiler) are still gated by the parent organization.

Where does CockroachDB fall on this spectrum? Currently, a bit in the middle: the project is a bit more open to external influence than Go, but less so than Rust.

Linguistic clues that distinguish hybrid from non-hybrid FOSS projects

Term used in non-hybrid FOSS projects Term/activity in hybrid FOSS projects
"the software", "the project" the product
"project relevance" "product-market fit"
"advocacy", "publicity" "marketing", "advertisement"
"community management" "public relations"
"developer bandwidth" "human resources"
"development time" "development effort"
"commercially licensed fork" "enterprise edition/version"
"steering group" "product management"
"parent- or funding organization" "the company behind"
"private mailing list" "internal mailing list"
"feature creep" "mismanaged feature scope"
"user testimonials" "use case", "success story"
"seeking contributors" "hiring contributors"
"developer meetup" "company event"
"IRC" "Slack"
"periodic project update" "product newsletter"
"logo" and "identity" "brand" and "trademark"

CockroachDB alongside other FOSS projects

Organizational feature FreeBSD LibreOffice PostgreSQL Rust LLVM GCC Go MySQL RocksDB CockroachDB
FOSS license (✓)
Reproducible build and installation instructions (✓)
Public contributor guide
Community code of conduct
Core developers active on public forums (✓) (✓)
Public can chime in during steering discussions (✓)
Public archives of all steering discussions
Public process to gain steering rights (✓)
Public apprenticeships and learning resources for external contributors (✓) (✓) (✓)
Person(s) serving as community manager (✓)
Job board shared between funding organizations
Public events where users meet developers (✓) (✓)
Disclosure policy and public security advisories

Looking in the crystal ball

CockroachDB is still a young project and its governance is bound to evolve as the company behind, Cockroach Labs, grows. It has only been three years! Consider that when they were three years old, only few people knew about PostgreSQL, MySQL, Rust, Go or LLVM.

Meanwhile, a few predictions seem possible. In the case of Mozilla's Rust and Google's Go, the parent organization's primary income comes from other sources and the FOSS projects were able to slowly grow at their own pace, on a research budget. As a result, their governance could grow organically as a balancing act between the needs of envisioned applications inside the parent organization and those of the community of users and external contributors (albeit slightly more so in the Rust's case than Go's due to different engineering cultures and project priorities). In contrast, Cockroach Labs seeks to leverage CockroachDB as its primary source of income, so one can reasonably expect that the project's governance will strongly privilege Cockroach Labs over its community, at least until the business is established.

Meanwhile, the steady expansion of online resources for external contributors (e.g. the last year saw the appearance of detailed architecture docs, contribution tutorials, and public documentation for issue triaging and prioritization) suggest that Cockroach Labs does intend to become more welcoming, and intends to increase its support of a community of open source developers. That said, whether and how this will extend to openness in the steering processes remains to be seen.

(Disclaimer: despite my involvement with the project, I am not privy to the specific plans regarding Cockroach Labs' FOSS community management and outreach. The statements and predictions enclosed in this article are personal opinions and do not reflect the plans or commitments of Cockroach Labs.)

Find me on twitter!

Copyright © 2018 Raphael ‘kena’ Poss. Permission is granted to distribute, reuse and modify this document according to the terms of the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/.