This post reflects a bit on 2022 and contains wishes for 2023. It mixes project and technical stuff, which I write about fairly frequently, and more personal reflections, which I don’t.

Reflections on 2022

Overall 2022 was a good year for me, though it was stressful and challenging in some ways. A lot of the stuff I did I’ve talked about in this blog, but there are a few other things, and quite a few more things in the pipeline. It was not a year of shipping anything big, and that is one thing I’d like to see different in 2023.

What I do straddles a number of more traditional categories. I do research, but most of my output is blog posts and code, not academic papers. My area of focus is primarily 2D graphics and GUI infrastructure, and these are to a large extent neglected step-children of academic computer science - there aren’t conferences or journals that specialize in the topic, nor are there decent textbooks. As a result, knowledge tends to be somewhat arcane, and just as much “lore” shared among practitioners as a literature. Even so, I find the field intellectually very stimulating and consider this odd situation to be an opportunity.

My portfolio of projects is very ambitious, as it basically includes an entire Rust UI toolkit plus a lot of the supporting technologies for that. It’s arguably too much for one person to take on, but I’m trying my best to make it work, largely by fostering a community around the projects. In late 2022, a big step was setting up weekly office hours, one hour a week, where we check in and discuss the various projects. I think that’s working well and look forward to continuing it.

On happiness

I’m not one of those “quantified self” people, but I have noticed that my happiness tends to correlate pretty directly with how much code I’m writing. I’m sure some of that is simply because if there’s stressful stuff going on that gets in the way of coding, that makes me unhappy, but obviously I just really enjoy it.

In particular, I love solving deep puzzles, and I find plenty of opportunity for that. An especially enjoyable track is adapting algorithms to be massively parallel so they run efficiently on GPUs (especially compute). Sometimes that leads to friction; my projects often have a “rocket science” nature to them, making them hard to contribute to.

Aside from puzzle-solving, which is largely a solitary activity, I also like the aspects of teaching, getting people up to speed, especially in topics that are not easily accessible through a standard computer science education or textbooks. Writing, including this blog, is a big part of that.

To a large extent, I will try to make time in 2023 to focus on these things I really enjoy, that I find make me happy. I am extremely fortunate that my paying job, as a research software engineer on Google Fonts, lets me do that.

Vello

Of the various projects in flight (and there are many!), one is clearly rising to the top of the stack: Vello, the GPU-accelerated 2D renderer. We made a lot of progress last year, but it’s still not quite ready to ship. Part of it is that it’s trying to solve a very hard problem, and part of it is that some solid GPU infrastructure needs to be in place for it to fly. A lot of time and energy last year went into GPU infrastructure in various ways, both pushing piet-gpu-hal forward, and then doing a full rewrite into WGSL and WebGPU (writeup to come).

For those who might be confused by the name, Vello is a rename of piet-gpu but still fundamentally the same project. The old name didn’t really fit, as “Piet” refers to a trait/method abstraction for the traditional 2D graphics API, and we’re moving away from that. The new name is intended to evoke both vellum (parchment, as in books and manuscripts) and velocity.

It is clear to me that there is strong demand for a good, cross-platform 2D rendering engine. There are a few other interesting things going on the space, but I’ll be honest, what I see, I want to compete with. There are big missing pieces (the most important by far is the ability to import images), but I see a pretty clear path to getting all that done.

So this is by far the biggest goal for the year: get Vello to a usable state. That involves shipping a crate, doing a proper writeup, which will probably be a 20-30 page self-standing report, and doing quantitative performance evaluation against other renderers.

Xilem

Another major initiative is Xilem. This was originally just a reactive layer, intended to be generic over the underlying widget tree, but is emerging as an umbrella for the larger project.

My hypothesis is (still) that Xilem is the best known reactive architecture for Rust. If I am correct, that means it is more concise, more ergonomic, more efficient, and better integrated with async than comparable work on Dioxus, Sycamore, pax-lang, and the Elm variants. That would be an exciting result, in which case I hope and expect some interesting systems will be built on the architecture. If I am wrong (which is entirely possible), then those other reactive approaches are all likely good enough for practical use, and have more evidence behind them than Xilem currently does. I feel strongly that it is worthwhile to test the hypothesis, and the only way to do that is to try to build real systems on the architecture. That experience will also likely drive improvements to the Xilem architecture, and I expect we’ll learn something interesting in any case. [Note: this paragraph is a rewrite of the original; see raphlinus#88 for the diff]

Progress on Xilem will have a considerably different texture than Vello. I hope that a huge fraction of the actual implementation work will be done by the community. Most of that work is most decidedly not rocket science, as I expect lots of it to be straightforward adaptation of the existing Druid widget set into the new architecture (and I’m making certain decisions explicitly to make that easier). I’m trying hard not to lick the cookie too much, and encourage other people to take on subprojects. I’m also trying to foster a culture where everyone in the community feels empowered to review PR’s and keep things moving, as having that block on me has been difficult.

One thing I am looking forward to working on is immutable data structures for efficiently (sparsely) diffing collections, which will hopefully realize the promise of my RustLab 2020 talk. This is a problem I feel has never been fully solved in UI toolkits - either you use complex and fragile mechanisms to incrementally update the UI, or you end up diffing the whole collection every time - and I believe using solid computer science to solve it, plus good Rust API design, would be very satisfying.

I consider Xilem to be more speculative and riskier than Vello. The extent to which it succeeds is largely based on how well the community can organize around getting the work done; if it gated on me, it’s a good question when it would all get done, especially with other projects competing for my attention.

The best place to learn more about Xilem is my High Performance Rust UI talk. I go over the goals and motivations of the project, and there’s good Q&A at the end. There’s a bit more about the Rust language aspects, particuarly trying to do ergonomic API design, in my RustLab 2022 keynote. In any case, I will be writing more.

Obviously Xilem depends on having good 2D rendering, so clearly time spent getting Vello production-ready contributes toward the overall success of the project.

To the people who want a good GUI toolkit they can use right now: I apologize, and ask for patience. In the mean time, you might check out Iced, Egui, or Slint. Those are all pretty good right now, and continually improving. I personally find the end goal of Xilem to be extremely compelling, but even in the best case it will take a while to get there.

Curves and other research

Over the holiday break, I let my mind roam more freely than usual, and I found myself coming back to various problems in curves. Probably the most satisfying work was refining the Bézier curve fitting ideas (see kurbo#230 for the code, hopefully writeup coming before too long). I also have what I consider a very promising idea to improve hyperbeziers, which have been on the back burner for a couple years, and an idea for robust boolean path operations.

But probably the juiciest bit of work will be perfecting the path geometry parts in Vello. In particular, I have a fairly compelling prototype of a combined flatten + offset operation in terms of Euler spirals, which I feel can be implemented efficiently on GPU as a compute shader and integrated into the Vello pipeline. That will improve handling of strokes (to handle all the join/cap options), and also serve as the basis for stem darkening of font outlines. Fun fact: the parallel curve work I did a few months ago was motivated by trying to get a good GPU implementation, but ultimately I believe that particular work is best suited for creative vector graphics applications, while the Euler spiral approach is better suited for GPU.

Non-goals

A year ago, I said only half as a joke, that my main New Years resolution for 2022 would be to not write a shader language. I’m pleased to report that I have succeeded. I will renew that vow for next year as well.

In many ways, it would make sense to start designing a shader language. My process for writing compute shaders is basically to design them in a fictional high level language with operations like prefix sum, stream compaction, etc., then translate by hand into the much lower level WGSL (formerly GLSL). If a good high level language existed (Futhark is the closest of what I’ve seen so far, but rust-gpu is also a contender), it would in theory streamline that work and let me write more efficiently.

But I think the advice in Don’t write a programming language is pretty sound. A programming language is a multi-year endeavor, and with a poor chance of success even in the most favorable conditions. In addition, I’ve found that everything in GPU land is at least 5 times harder than it is on the CPU. Case in point, I had trouble even getting Futhark to run on the GPU hardware of my main development machines, as it doesn’t have any compute shader back-ends, only OpenCL (which is basically dead) and CUDA (not much help on AMD or Apple M1 hardware). And that’s a relatively successful effort with over 8 years of experience!

I am also unlikely to write papers for academic journals and conferences. Perhaps it’s sour grapes, but I haven’t had a good experience, and feel that the (nontrivial) time and effort doesn’t really pay off. I’ll continue to use this blog, and publication of open source software, as the primary way I communicate research results. That said, I had one good experience of being asked to collaborate on an ASPLOS paper, and I remain open to collaborations. For someone with an incentive to publish academically, I think there’s quite a bit of raw material to work with.

Conclusion

I really feel that a lot of the research effort of the past year and beyond is poised to pay off in the next. Most especially, I am excited about shipping Vello, as I think that will advance the state of 2D rendering and also points the way for doing other interesting things in GPU compute. I also really look forward to working deeply with the community forming around Xilem to push that forward and hopefully make it reality.

There’s lots more that could be said, especially about the many individual subprojects, but in this reflection I hoped to give a general overview.

Discussion on /r/rust.

In any case, I wish everybody a happy and healthy 2023.