Rust extension traits, greppability and IDEs

Extension traits

This Rust RFC
provides a good, short definition of extension traits:

Extension traits are a programming pattern that makes it possible to add
methods to an existing type outside of the crate defining that type.

For example, here’s a trait with a single method:

trait Magic {
    fn magic_num(&self) -> usize;
}

We can now implement the Magic trait for our types:

struct Foobar {
    name: String,
}

impl Magic for Foobar {
    fn magic_num(&self) -> usize {
        if self.name.is_empty() {
            2
        } else {
            33
        }
    }
}

Now a FooBar can be passed wherever a Magic is expected. FooBar is a
custom type, but what’s really interesting is that we can also implement
Magic for any other type, including types that we did not define. Let’s
implement it for bool:

impl Magic for bool {
    fn magic_num(&self) -> usize {
        if *self {
            3
        } else {
            54
        }
    }
}

We can now write code like true.magic_num() and it will work! We’ve added
a method to a built-in Rust type. Obviously, we can also implement this trait
for types in the standard library; e.g.:

implT> Magic for VecT> {
    fn magic_num(&self) -> usize {
        if self.is_empty() {
            10
        } else {
            5
        }
    }
}

Extension traits in the wild

Extension traits aren’t just a fringe feature; they are widely used in the Rust
ecosystem.

One example is the popular serde crate, which includes code that serializes
and deserializes data structures in multiple formats. One of the traits
serde provides is serde::Serialize; once we import this trait and one of
the concrete serializers serde provides, we can do stuff like [1]:

let mut serializer = serde_json:: Serializer:: new(std:: io:: stdout());
185.serialize(&mut serializer).unwrap();

Importing serde::Serialize is critical for this code to work, even though we
don’t refer to Serialize anywhere in our code explicitly. Rust requires
traits to be explicitly imported to imbue their methods onto existing types;
otherwise it’s hard to avoid naming collisions in case multiple traits from
different crates provide the same methods.

Another example is the byteorder crate, which helps encode numbers into
buffers with explicit length and endianness. To write some numbers into a vector
byte-by-byte, we have to import the relevant trait and enum first, and then
we can call the newly-added methods directly on a vector:

use byteorder:: {LittleEndian, WriteBytesExt};

// ...

let mut wv = vec![];
wv.write_u16:: LittleEndian>(259).unwrap();
wv.write_u16:: LittleEndian>(517).unwrap();

The write_u16 method is part of the WriteBytesExt trait, and it’s
implemented on a Vec by the byteorder crate. To be more precise, it’s
automatically implemented on any type that implements the Write trait.

Finally, let’s look at rayon – a library for simplified data-parallelism. It
provides magical iterators that have the same functionality as iter but
compute their results in parallel, leveraging multiple CPU cores. The rayon
documentation recommends to import the traits the crate injects as follows:

It is recommended that you import all of these traits at once by adding
use rayon::prelude::* at the top of each module that uses Rayon methods.

Having imported it thus, we can proceed to use Rayon as follows:

let exps = vec![2, 4, 6, 12, 24];
let pows_of_two: Vec_> = exps.par_iter().map(|n| 2_u64.pow(*n)).collect();

Note the par_iter, which replaces a regular iter. It’s been magically
implemented on a vector, as well as a bunch of other types that support
iteration.

On greppability and code readability

All these uses of extension traits are pretty cool and useful, no doubt. But
that’s not the main point of my post. What I really want to discuss is how the
general approach relates to code readability, which is in my mind one of the
most important aspects of programming we should all be thinking about.

This Rust technique fails the greppability test; it’s not a word I made up –
google it! If it’s not immediately apparent, greppability means the ability to
explore a code base using textual search tools like grep, git grep,
ripgrep, pss or what have you.

Suppose you encounter this piece of code in a project you’re exploring:

let mut wv = vec![];
wv.write_u16:: LittleEndian>(259).unwrap();

“Interesting”, you think, “I didn’t know that Vec has a write_u16
method”. You quickly check the documentation – indeed, it doesn’t! So where is
it coming from? You grep the project… nothing. It’s nowhere in the
imports. You examine the imports one by one, and notice the:

use byteorder:: {LittleEndian, WriteBytesExt};

“Aha!”, you say, “this imports LittleEndian, so maybe this has to do with
the byteorder crate”. You check the documentation of that crate and indeed,
you find the write_u16 method there; phew.

With par_iter you’re less lucky. Nothing in imports will catch your eye,
unless you’re already familiar with the rayon crate. If you’re not, then
use rayon::prelude::* won’t ring much of a bell in relation to par_iter.

Of course, you can just google this symbol like this and you’ll find it. Or maybe
you don’t even understand what the problem is, because your IDE is perfectly
familiar with these symbols and will gladly pop up their documentation when you
hover over them.

IDEs and language servers

These days we have free, powerful and fast IDEs that make all of this a
non-issue (looking at Visual Studio Code, of course). Coupled with smart
language servers, these IDEs are as familiar with your code as the compiler;
the language servers typically run a full front-end sequence on the code, ending
up with type-checked ASTs cross-referenced with symbol tables that let them
understand where each symbol is coming from, its type and so on. For Rust the
language server is RLS, for Go its gopls; all popular languages have them these
days [2].

It’s entirely possible that using a language like Rust without a sophisticated
IDE is madness, and I’m somewhat stuck in the past. But I have to say, I do
lament the loss of greppability. There’s something very universal about being
able to understand a project using only grep and the official documentation.

In fact, for some languages it’s likely that this has been the case for a long
while already. Who in their right mind has the courage to tackle a Java project
without an IDE? It’s just that this wasn’t always the case for systems
programming languages, and Rust going this way makes me slightly sad. Or maybe
I’m just too indoctrinated in Go at this point, where all symbol access happens
as package.Symbol, packages are imported explicitly and there is no magic
name injection anywhere (almost certainly by design).

I can’t exactly put my finger on why this is bothering me; perhaps I’m just
yelling at clouds
here. While I’m at it, I should finally write that post about printf-based
debugging…