HN Reader

Back in the 90s, I implemented precompiled headers for my C++ compiler (Symantec C++). They were very much like modules. There were two modes of operation:

1. all the .h files were compiled, and emitted as a binary that could be rolled in all at once

2. each .h file created its own precompiled header. Sounds like modules, right?

Anyhow, I learned a lot, mostly that without semantic improvements to C++, while it made compilation much faster, it was too sensitive to breakage.

This experience was rolled into the design of D modules, which work like a champ. They were everything I wanted modules to be. In particular,

The semantic meaning of the module is completely independent of wherever it is imported from.

Anyhow, C++ is welcome to adopt the D design of modules. C++ would get modules that have 25 years of use, and are very satisfying.

Yes, I do understand that the C preprocessor macros are a problem. My recommendation is, find language solutions to replace the preprocessor. C++ is most of the way there, just finish the job and relegate the preprocessor to the dustbin.

> the "header inclusion" way is an O(N²) algorithm

Maybe I don't know what they mean by this, but the header inclusion way is O(N*M), is it not? Where N is the number of source files and M is the average number of dependencies each has (as expressed as #includes).

If M is approaching N -- meaning everything depends on everything -- you probably have bigger problems than compile times.

That's also setting aside precompiled headers. Those wouldn't be N^2 either, even when the precompiled header needs to be precompiled.

All that the C++ committee needed to do was just introduce "import" as "this is the same as include except no context can leak into it".

Would have been dirt simple to migrate existing codebases over to using it (find and replace include with import, mostly), and initial implementations of it on the compiler side could have been nearly identical to what's already there, while offering some easy space for optimizing it significantly.

Instead they wanted to make an entirely new thing that's impossible to retrofit into existing projects so its basically DOA

The standardization process here feels similar to what happened with JavaScript modules. Introduced in ES2015, but the language standard only had syntax and some invariants. It had no notion of how to load modules, or how they might be delivered, or how a program that had a module at its root might be started. But there was a similar urgency, of "we must have this in ES2015".

I made it one of my first projects after joining the Chrome team to fix that gap, which we documented at [1]. (This reminds me of the article's "The only real way to get those done is to have a product owner...".)

You could even stretch the analogy to talk about how standard JS modules compete against the hacked-together solutions of AMD or CommonJS modules, similar to C++ modules competing against precompiled headers.

That said, the C++ modules problem seems worse than the JavaScript one. The technical design seems harder; JS's host/language separation seems cleaner than C++'s spec/compiler split. Perhaps most importantly, organizationally there was a clear place (the WHATWG) where all the browsers were willing to get together to work on a standard for JS module loading. Whereas it doesn't seem like there's as much of a framework for collaboration between C++ compiler writers.

[1]: https://blog.whatwg.org/js-modules

While I can see compiler authors not wanting to have to turn the compiler into a build system, I'd really appreciate if they did do that. Having to create makefiles or other build artifacts is such a waste of energy for most applications.

Modules are unusable in real projects with real dependencies. Until that changes there’s no chance I’ll look at them.

If you avoid circular refs and forward declarations by writing hierarchical code, you won't really need modules. In my current project I'm 20+ headers in and still haven't had a circular ref. Just refactor all the commonly used code, decouple with callbacks and you have yourself a nice clean header only library that you can amalgamate and/or precompile.

> modules should be killed and taken out of the standard.

Yes. The C++ standard is an abomination that is both over-specified and wildly under-specified.

Modules are simply never going to work en masse. It is a failed idea.

had me at What to do with C++

what to do with them: try them, figure out that they're not supported by pretty much anything and thit when they do, they suck. Then move on with your life.

He makes such a good point. This is such a tragedy. Import std would of been an amazing thing that would of led some much less pain, but the committee got greedy.

The sensible way to speed up compilation 5x was implemented almost 10 years ago, worked amazingly well, and was completely ignored. I don't expect progress from the standards committees. Here it is if you're interested: https://github.com/yrnkrn/zapcc

The next major advance to be completely ignored by standards committees will be the 100% memory safe C/C++ compiler, which is also implemented and works amazingly well: https://github.com/pizlonator/fil-c

Why bother? The world seems to have moved on to Rust, C++ is only for legacy maintenance stuff anymore.

"You might want to grab a cup of $beverage before continuing, this is going to take a while."

Instead, I launched a compilation.

Fully agreed. The modules are an afterthought, and inevitably lead to a mess where we have both modules and includes all over the place.

Modules provide more than just speed. Compile time benefits are great and the article is right about build-time bloat being the bane of every developer. But modules serve a deeper purpose.

Explicit sub-unit encapsulation. True isolation. No more weird forward declarations, endlessly nested ifdef guards, or insane header dependency graphs. Things just exist as they are separate, atomic, deterministic and reliable.

Modules probably need a revision, and yes, adoption has been slow, but once you start using modules you will never go back. The clarity of explicitly declared interfaces and the freedom from header hell fundamentally changes how you think about organizing C++ code.

Start a new project with modules if you don’t believe me. Try them. That is the largest barrier right now - downstream use. They are not supported because they are not used and they are not used because they are not well supported. But once you use them you will start to eagerly await every new compiler release in hopes of them receiving the attention they deserve.

I did C++ for over 10 years, and now have been doing rust for about 4. On the whole, I like rust much better, but I really miss header files.

Modules are horrible for build times. If you change an implementation (i.e something that would not normally involve editing a header) the amount of rebuilding that happens is crazy, compared to any C++ project that was set up with a minimal amount of care.

They had an opportunity to finally get rid of requiring forward declaration, but they failed..

C++ has no future

My belief is that modules were designed for big tech companies that do massive builds parallelized across thousands of machines. Such an environment makes the complexity bearable and you get massive speedups from cached build artifacts. I think smaller companies and hobbyists kinda got screwed.

At Waymo we use c++ modules via clang and got the demanded 5x speedup.

As the article mentions, you need a close relationship between the compiler and build system, which Google already has. The google build tooling team got modules to mostly work but only turned them on in limited situations. But we but the bullet and turned them on everywhere, which has sped up compilation of individual files by more than 5x (I forget the exact number).

The remaining problem is that sometimes we get weird compilation errors and have to disable modules for that compilation unit. It's always around templates, and Eigen has been gnarly to get working.

Obligatory: https://arewemodulesyet.org/tools/

Not great!

I need more background and foundational insights for C++ control flow syntax for a pythonista who knows c like: I++ do {} while switch () case.

I absolutely love having headers in C and stronger notions of compilation units. But in C++ you tend to want to push everything into headers to take advantage of generics. This is another problem where a separate code generation step would have been superior to language generics

The problem with C++ modules is that they are empirically inferior to headers and separate compilation units.

The pre processor and linker, as derided as they are, allow for scaling of software size to extremes not possible with supposedly superior languages’ supposedly superior module systems.

Want to build a >billion line of code OS? You can do that with headers and separate compilation units. Good luck doing that with any other tech