HN Reader

Hah, I wrote the crypto/rsa comments. We take Hyrum's Law (and backwards compatibility [1]) extremely seriously in Go. Here are a couple more examples:

- We randomly read an extra byte from random streams in various GenerateKey functions (which are not marked like the ones in OP) with MaybeReadByte [2] to avoid having our algorithm locked in

- Just yesterday someone reported that a private ECDSA key with a nil public key used to work, and now it doesn't, so we probably have to make it work again [3]

- Iterating over a map uses a randomized order to avoid exposing the internals

- The output of rand.Rand is considered part of the compatibility promise, so we had to go to great lengths to improve it [4]

- We discuss all the time what commitments to make in docs and what behaviors to disclaim, knowing we can never change something documented and probably something that's not explicitly documented as "this may change" [6]

[1]: https://go.dev/doc/go1compat

[2]: https://pkg.go.dev/crypto/internal/randutil#MaybeReadByte

[3]: https://go.dev/issue/70468

[4]: https://go.dev/blog/randv2

[5]: https://go.dev/blog/chacha8rand

[6]: https://go-review.googlesource.com/c/go/+/598336/comment/5d6...

Solution to the specifically mentioned problem: Don't use string-based errors, use sentinel errors [1].

More generally: Don't produce code where consumers of your API are the least bit inclined to rely on non-technical strings. Instead use first-level language constructs like predefined error values, types or even constants that contain the non-technical string so that API consumers can compare the return value againnst the constant instead of hard-coding the contained string themselves.

Hyrum's Law is definitely a thing, but its effects can be mitigated.

[1]: https://thomas-guettler.de/go/wrapping-and-sentinel-errors

Hyrum's Law is one of those observations that's certainly useful, but be careful not to fixate on it and draw the wrong conclusions. Consider that even the total runtime of a function is an observable property, which means that optimizing a function to make it faster is a breaking change (what if suddenly one of your queues clears too fast and triggers a deadlock??), despite the fact that 99.99999999% of your users would probably appreciate having code that runs faster for no effort on their part.

Therefore it's unavoidable that what constitutes a "breaking change" is a social contract, not a technical contract, because the alternative is that literally nothing is ever allowed to change. So as a library author, document what parts of your API are guaranteed not to change, be reasonable, and have empathy for your users. And as a library consumer, understand that making undocumented interfaces into load-bearing constructs is done at your own risk, and have empathy for the library authors.

It's sort of Hyrum's Law but it's really just Go being Go. The error could've been an enum type that could be changed with only a string replace for consumers. Instead they are using strings as types, so now you have no idea how consumers might rely on it. They could check the middle 6 chars of the error and break if you change it. It's another terrible anachronistic design decision when better alternatives have been in use in other languages for decades. Early mistakes + inability to change things means you're stuck forever.

An interesting topic is how to fight Hyrum's law. A possibility is to add randomness in things you don't want people to rely on. If I remember well, this is what the QUIC protocol does. Some fields are unused in the current version, but required by the specification to be set to random values, not null bytes, so that routers don't start relying on them to identify the packets.

EDIT.

I think I found the source: https://www.rfc-editor.org/rfc/rfc9000#section-17.2.1

> The value in the Unused field is set to an arbitrary value by the server. Clients MUST ignore the value of this field. [...] Note that other versions of QUIC might not make a similar recommendation.

I think they call it "greasing", to prevent "ossification".

This is a good example of "stringly typed" software. Golang designers did not want exceptions (still have them with panic/recover), but untyped errors are evil. On the other hand, how would one process typed errors without pattern matching? Because "catch" in most languages is a [rudimentary] pattern matching.

https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...

At one job, I found a misspelling in an error message and fixing it only to discover that the web of dependencies on that misspelled text was so deep that it was impractical to fix and had to return to the misspelled text. It still bugs me.

Perhaps some package authors are more accepting of this than others. I stumbled upon this comment in the `json` package the other day:

// isValidNumber reports whether s is a valid JSON number literal. // // isValidNumber should be an internal detail, // but widely used packages access it using linkname. // Notable members of the hall of shame include: // - github.com/bytedance/sonic

It's interesting that this law is the exact opposite of the Robustness Principle / Postel's Law.

> be conservative in what you send, be liberal in what you accept

If you are liberal in what you accept, you'd better understand the ways in which you've been liberal, and document them (at least) internally, because you're going to have to support all those ways forever, even after huge codebase changes, due to Hyrum's Law.

I try to avoid creating APIs which are "liberal in what they accept" for exactly that reason.

what I learned from shipping APIs:

1. Clients will do whatever they need to do get their job done, even if it's not the publisher's intended way

2. Clients don't read documentation

3. Bugs will become part of API once enough clients rely on their behavior

4. The number of API calls does not necessarily equate to importance.

---

As such, I aim for the following when developing an API

1. Ship the beta API early and see how they use it to minimize the surprise. (This may not always be possible)

2. In most cases, bump up the major version while supporting the previous version. This means you'll need to define SLA for your API

3. Most clients are OK with breakages as long as they are given enough time to migrate, or the API provider gives them a tool to auto-migrate the code (if that's possible in your product)

Fred Brooks discussed this in the unfortunately named pun "The Mythical Man-Month". Most of the gray beards have read it, ask to borrow it, it will make their day. The punchline was on the IBM 360 they stopped fixing bugs when the fix cause the same or more bugs than the unfixed bug, which soon became all bugs.

Well aware of Brooks, when the loop var semantics were changed Go did an analysis showing that many more bugs were fixed than created by the change.

In Docker's error response for `docker rmi'; the fifteenth word is "container" and the sixteenth is the container ID.

Weren’t there a couple of anecdotes where Windows couldn’t fix a bug because some popular game (maybe SimCity?) depended on it, so the devs hardcoded a SimCity check inside Windows and made the bug happen if it was running?

Quite interesting, thank you.

However, in this specific instance, even if the text cannot be changed, couldn't the error itself in the server be processed and signaled differently, eg. by returning a Status Code 413[1], since clients ought to recognize that status code anyway?

[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/413

When I clicked on the link to codebases relying on the specific error string, I was expecting to see random side projects. Wasn't expecting to see Grafana and Caddy on the list.

Go seems really sensitive to this subject. Maps iterate in order, but one day they said “this is incidental and we said not to rely on it. You do, so we’re breaking it in a minor release” and now maps iterate in order… from a random offset

Clojure manages to do improvement to the language, without breaking the users.

Most of improvements are "additions", it is never a "change" or "re-do something better"

It is an awesome experience be able to upgrade the language anytime with no fear or pain.

Related XKCD: https://xkcd.com/1172/

Many occurrences of Hyrum's are "desire paths[1]" of APIs. For many people, the most obvious way to determine if the error was a MaxBytesError was to check the string. I'm not familiar with Go, but assuming it has RTTI, I'm guessing the intended path for people to take was to check the type of the error against MaxBytesError, and occurrences of this were people who either didn't know that, or found the error string to be immediately available in their tooling, but the type not immediately available.

[edit]

Per [2], this looks desire paths is even more an apt analogy than I thought; until 3 years ago, this code returned a generic Error type.

1: https://en.wikipedia.org/wiki/Desire_path

2: https://news.ycombinator.com/item?id=42202472

I wrote "// Due to Hyrum's law, this text cannot be changed." AMA.

Hyrum's Law especially applies when you have consumers of your APIs that violate Postel's Law. To minimise those in the past, we've introduced intentional jitter in our API responses that while didn't violate the schema prevented unintentional reliance on behaviour that wasn't intentional[1].

[1]: <https://medium.com/pageup-tech/update-on-driving-client-resi...>

IME, one of the places hyrums law shows up the most is in tests.

I've seen tests of various types (unit, integration end-to-end) break because they made assumptions about behaviors that weren't garanteed, and supposedly backwards compatible updates broke them. Here are some examples of things that have broken tests:

- an update resulted in a change in the order of elements in a Hashmap or set.

- a change in an error message (or other user-facing message), changed

- a change in how leap days are handled for datetime arithmetic

- change in the format of locale-specific datetimes

- the timezone offset for a given area

- removal of internal-only APIs that were accessed using reflection

- something performed faster, which revealed race conditions in the testing code

- changing the precise representation of some data format. To give a specific example, changing a single byte when gzip compressing a file, that has no impact on the compressed content.

Does anybody else always read this as Hyrule's Law?

Sure... but this is why we have sem versioning and release notes. It's always nice to try and support all users but sometimes you just need to ship breaking changes...

Another related effect of this is Protocol Ossification [0] which happens when implementers of a public API/Protocol surface area take implicit dependencies on common but not standardized behaviors of the API/Protocol implementation.

That being said, you can take proactive steps to defeat this. For example, the default Hash for strings in .NET is randomly seeded each time a process starts[1] in order to strongly dissuade folks from taking an implicit dependency on the underlying algorithm which is not guaranteed to be stable

[0] : https://en.wikipedia.org/wiki/Protocol_ossification

[1] : https://andrewlock.net/why-is-string-gethashcode-different-e...

This is why we have semantic versioning.

I think Hyrum's law really depends on APIs not applying consequences to people that depend on non-guaranteed behavior. The world needs more consequences for poor behavior.

Just randomly change the non-guaranteed stuff in every release and this behavior likely would stop and/or you'd lose the users that don't know any better. Both sides of that sound like a win to me.

The only reason we care about this law is that the end users matter.

If we can change a misused API such that things break only for the misusing developer, then that is usually a who-cares.

Sometimes that developer is in the same organization that you're in, and their users are your users.

Sometimes that developer is no longer in your organization and their code is your code.

There are cases when you need to make a choice if you want to fix the bug as it might break many people who rely on it. There is no real good answer but to be able to look forward and anticipate the misuse.

> so per Hyrum's Law it's probably relied upon by some.

Yikes. this kind of defensive posture with respect to Hyrum's law is extreme and absurd. Per Hyrum's Law everything is potentially relied upon by someone, keeping stuff that may be relied upon means you cannot change anything (see this infamous xkcd on this[1])!

Thinking that no change is acceptable at all isn't the right take-away from Hyrum's Law: instead you should be ready to have to roll back changes that break people's workflow even when you didn't expected the change to break anything (and it also means that you need to have a way for your users to communicate their issues to you, which definitely isn't something Google is well-known for …).

[1]: https://xkcd.com/1172/

It's like an inverted game of cat and mice

1 - Lang/OS/Lib developer puts out a quirky or buggy API (or even just an ok API)

2 - Developers rely on a quirky, weird or unexpected side effect because it's easier/more obvious or it just works this way due to a bug

3 - Original developer can't fix it because it would break compatibility

4 GOTO 1

Corollary: uptime is part of the defacto spec being relied on.

One of the SRE practices is breaking your service on purpose to bring the actual service level closer to what is promised and supported.

I wish someone in the Chrome team knew about Hyrum's Law before they released a breaking change in the way chrome validates custom URLs.

At work we have a data providers whose API always returns 200, often with the text content “ERROR: …”

And that’s how put Hyrums law into effect.

One interesting metric for LLMs is that for some tasks their precision is garbage but recall is high. (in essence: their top 5 answers are wrong but top 100 have the right answer).

As relates to infinite context, if one pairs the above with some kind of intelligent "solution-checker," it's interesting if models may be able to provide value across absolute monstrous text sizes where it's critical to tie two facts that are worlds apart.

People who parse text error messages deserve what they get

Should this not be handled by checking "resp.status == 413" ?

What do they call the law that says they can’t increase the major version?

> to design systems in a way that minimizes the chances of unintended

go team did their best to define a custom MaxSizeError to discourage developers from the flimsy string dependency.

Every system hits a limit on the amount of guard rails and protections needed to protect foolish customers from their own bad behavior.

Sometimes you need to deliberately break dependencies that were never meant to exist to reveal the vulnerabilities in a system.

like https://en.wikipedia.org/wiki/HTTP_referer can't be referrer ever

This line of the article resonates with me a lot: > A good reminder to be careful when changing code others might depend on

If you maintain a widely use Free Software library, please consider avoiding breaking changes when possible.

I'm not going to imply you have any obligation to do so, but your users will appreciate it.

see also https://news.ycombinator.com/item?id=16286764

I don't know. I think Hyrum's law should not prevent the project from advancing. If the user is so dependent on non-contact behavior of the API, they have to expect that some logic might break even on minor version updates.