Giving up on Open Source Software
Recently, I was quoted in an interview at uses this saying
I decided to stop using open source software whenever possible and use any available alternatives…
After publication, I received an inquiry asking me to explain my reasoning — and this essay is my explanation. This explanation is narrowly focused on the question of using open source software — which is distinct from my ideas on writing open source software or distributing open source software or reading open source software or modifying open source software or copying open source software. Furthermore, the using which is at issue is running it, as opposed to using it by having it be a dependency in a software project I’m building. This is about being a user of software, not a builder thereof. There are good reasons for considering open source software if you are building software. I’m not sure the same is true if you are using software.
I also want to point out that I was, in the past an open source software enthusiast, and spent many years preferring it to proprietary software. I had a lot of open source software installed at one point. However, the world changes, and what may once have been appropriate is not guaranteed to be so forever. And so the time came when I decided to consider the question: Should I continue to use open source software? In mathematics, if you want to prove a hypothesis, you start out by trying to prove (or assume) the opposite hypothesis, like: prove that the square root of two is irrational. The proof begins thusly: Suppose that the square root of two is rational… So, if I wanted to test the hypothesis: using open source software is better than using proprietary software, I would start as follows: Suppose that I switched to using only proprietary software… One possible outcome is that I would then discover a set of reasons why this was a bad idea. Another possible outcome is that I would discover why this was a good idea. Either way, I would learn something. And then, I could construct some theories to explain what I had observed.
In 1937, Ronald Coase wrote an essay (The Nature of the Firm) in which he considered the question: Why do companies exist? If we have a free market economy where people can exchange their services for money, what is the difference between employees (who exchange their services for money) and contractors (who exchange their services for money). His conclusion was that the difference was in the transaction costs: forming a company and having employees reduces the transaction costs in the exchange of labor for money over using a contractor model. Companies like Uber exist, presumably, because they figured out a way to lower the transaction costs for contractors to the point where it didn’t make any sense to have them be employees.
A similar logic applies to open source software. If the software is free for me to download and configure and build, then it is equally free for someone else to download and configure and build. If there were effort involved in building and maintaining it, it would only make sense for me to do it myself if the transaction costs of having someone else do it exceeded some threshold. Twenty years ago, for most software, that threshold was exceeded: software was mostly pretty expensive. As the cost of building and distributing software came down, the number of software packages for which it made sense to pay someone else to manage it increased. Red Hat’s business model was always that: it would be cheaper to pay someone else to maintain the software than to do it yourself. Having taken the decision to pay someone else for the software, the “open sourceness” of it is their problem (or advantage), not yours. You are specifically paying them so you don’t have to deal with the source code.
In fact, the original GNU manifesto argues that this would be the business model for software developers to make a living: that they would charge a fee for distributing or maintaining software. Quoting from that manifesto:
If it is true that users would rather spend money and get a product with service, they will also be willing to buy the service having got the product free.…
Some users may obtain copies at no charge, while others pay to obtain copies
As someone who used to make his living writing software, I would like for there to be profitable software business models. And given Coase’s theorem, if the transaction costs are sufficiently low, then the cost for me to build, deploy, troubleshoot, etc. my software would be equal to the cost for me to pay someone else to do it (always assuming efficient markets). Given my desire to help aspiring software workers, and my limited resources, I might prefer to (say) do my own yard work and pay someone else to manage my software, rather than manage my own software and pay someone else to do the yard work. Doing my own yard work has the additional benefit of exposing me to fresh air and getting some exercise.
One way in which the world has changed is in the pricing of software. Twenty years ago, the idea of “free trials” was a very niche idea. Indeed, it had its own name: shareware. Software which could be tried out before purchase was called shareware — and it was mostly games or small utility programs. Pricing models for software have become much more sophisticated in the last twenty years. Software pricing is different for individual users than for corporations. Most software has some kind of free trial. Lots of software can be used for free at an introductory level, with pricing tiers for advanced functionality or increased usage. Years ago, each copy of proprietary software had to be licensed. Today there are family plans, most software can be run on two or three or five computers on a single license. For many pieces of software, the cost of proprietary software is the same as for open source software. For others, the pricing is so low that it may be worth it for the inconvenience of dealing with the source code. For most software, there is no cost advantage to choosing open source. My canonical example here is text editor. Once upon a time I used Emacs (open source). Then I switched to Vi (Also open source). Then I switched to TextMate (proprietary — which I happily paid for). Then, they went out of business, and released the software as open source. So I switched to Sublime Text (proprietary — which I paid more for, less happily — but happily enough). Why did I switch? Because the conversion of TextMate to open source was proof that they could not afford to continue to upgrade the software and provide new features and fix problems. And finally, I switched to BBEdit (proprietary — which I needn’t pay for unless I want the advanced features — which as of this writing, I have not yet needed). For all the software that I use, the pricing of the proprietary vs. open source versions was not an issue — except for one application. That application is Blender. The proprietary alternatives are priced in the thousands of dollars, not the tens of dollars, and that threshold makes the available proprietary versions unaffordable to a hobbyist. I am told that there are advantages to the proprietary packages, but as a dilettante in 3-D modeling, I cannot conceive of any feature which would persuade me to part with that much money. So, aside from Blender, there is no financial advantage to using open source software over proprietary software.
Let me meander into a brief digression. I noticed just recently that Homebrew, which used to be a means of installing open source software on a Mac, had at some point in the last few years, become a mechanism for installing software on a Mac that didn’t care whether it was Open Source or not. The new “brew casks” command lists over three thousand packages that can be installed. I looked at three at random, and all three were proprietary software which would have been called Shareware back in the day. It is still possible to build and install from source, but there are now options to configure homebrew to never use the source code. When open source tooling has evolved to the point where it installs proprietary software and open source software without any indication as to which is which — and provides an option to never try to build from source — I feel comfortable in thinking that other people have started to notice that the “open sourceness” of software is no longer very relevant. And why was I looking at Homebrew? Because of another trend in the software ecosystem which I’m going to need to digress from this digression to explain.
I recently bought a new coffee machine. A fancy espresso making coffee machine. It has a little touch screen that lets you select your drink — press for an Americano, or flat white, or cappuccino. There is probably software in there somewhere to display the menu, track the touch, and so on. There is no wifi or bluetooth support. There are no ports. If there is a way to load new software, I don’t know what that would be. It might not even be software. Back in the day, when you couldn’t patent software, Apple took their Quickdraw software, burnt it into ROM, and then patented the ROM. Because ROMs, of course, are not software, they are hardware. So, although you could say that there is software inside my coffee machine, you could also say that my coffee machine is hardware. Whether my coffee machine should be considered hardware or software is kind of a social construct. After all, I’m just guessing that there is software in there. But who knows, it could be electromechanical switches. What difference would it make?
In this context of looking at an appliance as a piece of hardware (and if you don’t have a fancy espresso machine, you can substitute dish washer or refrigerator or oven or thermostat or doorbell), there were family members over the years who had the same appliance conception of their mobile phones and laptops. For the entire life of the device, they never installed any software. It was an appliance that came with some functionality (which might be implemented via software, or maybe super-miniaturized electro-mechanical components), and so they used the devices for the functionality that they had when removed from box they came in. You may be sure I occasionally mocked these relatives as clueless Luddites.
However, when it came time to explore the issue of “is using open source software better than using proprietary software”, I realized that there was an even more fundamental question. Assuming that (as a social construct), the device that I remove from the box is a piece of hardware, then the more fundamental question is “is using software better than using hardware?” Now, I can hear some of you typing furiously to send me a comment pointing out that mobile phones and laptop computers (or desktop computers) are full of software. Please don’t. If I take a mobile phone and drop it on a hardwood floor. It makes a noise. If I grab a screwdriver (or whatever tool I need) to pry it open and look inside — everything in there is hardware. It is all hardware. The fact that you believe that there is software in there is a social construct. If it were all in ROM, then, like the original Macintosh computers, it would legally be hardware. Which proves that it is a social construct. If some group of people can get together in some contexts to argue that it is software, and other groups of people can get together in some other contexts to argue that it is hardware, then that is the very definition of a social construct. Quickdraw was software. Except legally, where it was hardware. And also logistically (ask anybody who needed to acquire parts to build a Macintosh clone). Maybe my relatives who thought mobile phones should be considered hardware weren’t clueless Luddites. Maybe I was the clueless one, and they were ahead of their time.
So, having accreted a bunch of software which I had installed post-unboxing on my iMac, the first step was to delete everything — restore it to its original unpolluted hardware state — and see if that was as effective as having installed software. Rather than factory reset, and perhaps cause a problem with something that relied on installed software, I proceeded to conduct the experiment by going through the installed software and delete each bit of software and see if the built-in hardware was sufficient for my purposes. Hence the foray into homebrew — as well as going through the Applications folder and deleting things that hadn’t been there in the box.
If you treat your device as a, well, device, then you don’t have to worry about acquiring software components, upgrading software components, dealing with software incompatibilities, and the like. In the case of some device manufacturers, they may have a service like a maintenance contract. I have maintenance contracts with gutter cleaners, an HVAC company, and yard maintenance. They come over and blow leaves out of gutters, change filters on my air conditioner, and trim the hedges. Oh, and I also have a maintenance contract on my mobile phone; the manufacturer occasionally vacuums out some bugs and polishes the user interface.
As I point out in https://usesthis.com/interviews/r0ml/ , on mobile devices, a large number of the “software” I install are “web page replacements” for services (like banking, or shopping, entertainment) which on a desktop computer would be a web page, but on a mobile device are a separate “app”. Of course, I never had any open source software on my mobile device, so the experiment of abandoning open source software was restricted to my desktop computer. A new iMac comes with a bunch of built-in functionality. Word processing, video-conferencing, calendar management, e-mail, spreadsheet, web-browsing, and so on. If one were an open source advocate, as I once was, one might decide that even though one’s desktop computer came with a built-in hardware web browser, one might install a software web browser like Firefox. and so I had. But as part of this experiment, the question was, if I removed the software web browser and used the hardware web browser, would there be a noticeable improvement (or degradation). And not just the web browser, but any other software for which there was a built-in hardware alternative. Thus began the slimming down of my software portfolio.
There is one more digression before summarizing the conclusions to this experiment. And that is explaining a phenomenon in software development which I call lithification. I describe this in my 2017 talk at linux.conf.au https://www.youtube.com/watch?v=i3nJR7PNgI4
Lithification is a term from geology which refers to the process by which loose materials are converted into coherent solid rock by compaction or cementation. In the software world, the analogous process is the one in which lots of little bits of functionality get absorbed into a larger software substrate. So, for example, there was a time when there were lots of little barcode scanner libraries and apps, and eventually android and iOS just built barcode scanning in as a feature of the operating system. In the 1980’s one would purchase a networking stack which was purchased separately because the operating system didn’t include networking. Somewhere along the way, networking software got built into the operating system. Go back further and things like support for sound cards involved separately installed drivers, since the operating system did not have built in sound support. More recently, things like machine learning, or virtual reality or raytracing are being cemented or compacted into the operating system. That’s lithification. It can happen with development environments — over the years they have had lithification occurring with features like version control and syntax coloring and integration with cloud deployment and so on. Each of these features at one time was a separate application which was eventually subsumed.
Open source is usually bad at this. (There are a few exceptions — it’s not completely hopeless.) And yet, I believe that this is the future of software. And here, once again Coase’s Theorem comes into play. Organizations that build computing platforms (Amazon, Microsoft, Google, Apple, Oracle, etc.) reduce the transaction costs of lithification. When they acquire startups they take the code from those applications or libraries, and through the application of metaphorical pressure, they compact and cement that functionality into the existing platform. The platform grows. They can do this because a market exists that enables them to acquire other bits of software.
Open source, by contrast, adopts a strategy of incorporating other software by reference — to create an external dependency. Each little bit of software has its own repo and dev team, and the transaction costs for integrating them remain high in perpetuity. There is no gravitational force or pressure acting to lithify all the loose materials. If the currency in open source development is, as Eric Raymond proposes, “reputation” (Homesteading the Noosphere), then I need to keep my reputation-generating software distinct from the software of others. If my software is absorbed into somebody else’s project, I lose my reputational credit. Aside from creating additional costs on the development and integration and debugging side, this morass of individual components creates cognitive load in trying to use all the components. A classic example is authentication. In a Microsoft platform, when you were logged in to the operating system, and then tried to access a SQL Server database, you were logged in to the database with your Windows account. And when you used Internet Explorer to access an intranet website hosted on IIS, you were logged in with your Windows account. This always made Windows easier to use and manage than Linux.
On a Macintosh computer or phone, if you’re logged into iCloud, you’re logged into iCloud in every application you’re using. Every application can share with every application using the same share facility. Keyboard shortcuts are the same everywhere (mostly).
The open source crowd think that having plugins and extensions and individually installed libraries is a benefit — it increases freedom. But it also increases cognitive load and incompatibilities and upgrade complexity. It increases software transaction costs. Eventually the costs get to the point where the best strategy is to retrench and use proprietary software which is lithifying. The evolution of separate applications into pre-installed hardware features is one of the ways in which lithification manifests. When Microsoft first attempted this with Internet Explorer, to lithify a web browser into the operating system, they were sued by the government. Today, every platform includes a built-in web browser. And other built in functions as well. Like photography. Video player. Text messaging. All of these were separately purchased applications at one time.
The impact of lithification is not just felt at the application level. It also impacts the software development process. If I use a modern platform, the platform includes data and video compression libraries, cryptographic libraries, image processing libraries. It includes interfaces for software installation and updates, for backups and authentication and networking and handwriting recognition. Not only does it make it easier for the developer to create software, but applications developed by different people or organizations will be consistent in ways in which applications developed using open source libraries will not. If I am using different software components that share these libraries, I reduce the cognitive load required to use them effectively. They are all installed the same way. They are all updated the same way. They all use the same keyboard shortcuts. They all use the same scripting language. They all transfer state between devices the same way. They all use the same spellchecker. They all allow you to use handwriting with a digital pencil in the same way. Some open source platforms (I’m thinking KDE and Gnome) have succeeded in doing this to some extent. I was a big KDE fan back in the day, because they did this better than any open source alternative. Almost as well as the major device manufacturers. I sometimes think that the early success of GNU was that all those early programs were built on commercial Unix libraries. Instead of building up from there, GNU then started replacing the underlying platform layers in a form of reverse lithification: dredging the oceans of the world to replace the layer under the sand. Eventually, they reached bottom, and foundered on the HURD. But I digress.
When I install Emacs I need to script it in LISP — which I must learn only for Emacs. When I install Vi I need to script it in viscript which (as its name suggests) I learn only for vi. When I install TextMate, I need to script it in Ruby, which helps me with no other software I use. When I install Sublime Text, I script it in Python, which Blender also uses. But when I install BBEdit, I script it in AppleScript, which is what every hardware application is scripted in. Once I invest the cognitive capital to use AppleScript, I can script, not only my text editor, but a host of other applications. Should there be an update, BBEdit will be updated automatically (as all my hardware applications will be). Emacs will require some other procedure for updating. Python is getting traction as a scripting library with many applications, but the hardware applications that are built into my device don’t use Python, and their number keep increasing through lithification. It is entirely possible that Apple might choose to use Python as their preferred scripting language. In which case, I would expect all of the pre-installed applications to be scriptable via Python, and all the commercial software vendors that use Apple development tools and libraries would all wind up being Python scriptable.
As with the early GNU, any open source application could be built on the existing libraries, but they mostly choose not to, preferring to include a host of potentially mutually incompatible non-hardware libraries. Without dwelling on the difficulties this introduces for the software developer, it introduces cognitive load and management complexity to the eventual user (myself). If I choose commercial software that has attempted to take advantage of common platform resources, I find that I simplify my experience, manage my cognitive load, reduce the frustration that software loves to scatter in its wake. If open source software were equally willing to take advantage of the common platform resources, it would be a level playing field. By declining to do so, open source software handicaps itself.
For something as simple as: I would like to have a single package manager — the use of open source software becomes problematic. Everybody agrees that this is a good thing. Linux distributions all began by providing this one bit of functionality as a major part of their value proposition (yum, rpm, apt). On the mac, as I pointed out earlier, homebrew has evolved from open source only packages to include non-open source packages as well. But the App Store is also a package manager to manage the installation of software. Why not just use that? In the case of GPL-licensed software, there is a general agreement that the license is incompatible with the App Store, in which case I would be disinclined to use that kind of software if an alternative were available. Other kinds of open source software do not have similar licensing restrictions to using the App Store, yet many choose not to. If they have chosen not to, that usually means that they have also chosen not to integrate AppleScript support, or iCloud support, or take advantage of the native GPU libraries, etc. It is a red flag that indicates that the software is likely to be disadvantaged in a number of performance and compatibility fronts. It also means that as the operating system continues to evolve and lithify additional functionality into itself, those software packages that disdain to advantage themselves of the hardware capabilities will constantly be falling behind those that do. Android users might experience the same dynamic with their ecosystem’s features, as might Microsoft users.
So, if I had pre-installed hardware that had the functionality that I need, I would just use that. Otherwise, I would go to the App Store and search for an app that would meet those requirements. I would decide by reading the descriptions, looking at the screenshots, searching for reviews on the web. I do not need to look at any source code to decide which one to use. That would take way too long. Even open source enthusiasts pick among various open source application alternatives by doing research that mostly doesn’t involve looking at the source code. (Did your decision to pick vi or emacs entail looking at the source code of both?) Having made the decision, I would install my choice by clicking the “Install” (for free software) or “Buy” (for commercial software) button. If the software did not meet my requirements, I wouldn’t have chosen it, and thus, would not need to download the source code so that I might modify it. If it did meet my requirements, I would not need to download the source code so that I might modify it. If it turned out that I was mistaken, and the software I downloaded did not meet my requirements, I would — rather than download the source code to modify it and rebuild it — download my second choice application and try to use that.
It turns out that there are a number of applications that I felt the need to install, over and above the existing hardware applications. Of those, only two are open source: Blender and Handbrake. I’m looking for a commercial alternative for Handbrake. And I’m looking for ways to reduce the number of software applications by learning more about my platform to see if I can accomplish the same objectives using only hardware. I have faith that if there are open source applications that are really useful, the platform vendors will lithify them into their devices, and turn them into hardware for me. But if they don’t, enterprising software entrepreneurs will lithify them into proprietary software applications that I can purchase. Either way, I don’t have to worry about open source software.
