PKGW

One Good Tutorial at HPC Best Practices Webinar: Slides

Mon, 16 Mar 2026 14:55:10 -0400

On Wednesday I’m going to present my scientific software documentation project, One Good Tutorial</a>, as part of the IDEAS</a> HPC Best Practices webinar series</a>. It’ll take place at 1 PM US Eastern time. Register to attend</a> if you’re able to watch it live, but you can always watch a recording later if you’re not. My slides are attached to this post.

Here are the slides</a> if you want to check them out.

I can say that as I’ve been polishing and promoting the project, I’ve focused more and more on the idea of a “minimum viable documentation product”. I think it really captures the core idea that One Good Tutorial is trying to convey, and hopefully it anchors that idea in a context that a lot of people are familiar with. There is a bit of potential acronym confusion</a>, though.

The work described in this post was supported by a Better Scientific Software Fellowship</a>.

One Good Tutorial: Beta Release!

Wed, 21 Jan 2026 22:18:48 -0500

I’m delighted to announce that a beta-testing version of my scientific software documentation resource, One Good Tutorial</a>, is ready for your feedback! Check it out at onegoodtutorial.org</a>. 
(You might have noticed that my blogging pace has fallen off a cliff. Having a kid will do that to you! I aspire to ramp things back up, but it’ll probably be a little while …)
One Good Tutorial is the guide that I’ve been developing over the past ~year thanks to the support of a Better Scientific Software (BSSw) Fellowship</a>. My proposal was all about helping maintainers of small-to-medium scientific software projects create better documentation with more confidence, and in my last update</a>, I laid out my first-draft vision for what I would build.
Based on some initial work surveying</a> the state of the field</a>, I came to some decisions about where I wanted to direct my focus. First, I wanted to hammer my target audience over the head with the idea that there’s more to docs than API docs (hugely influenced by Diátaxis</a> here). Second, I wanted try to help out people who want to do a good job with their docs, but don’t really know how to get started.
My initial concept was a “checklist matrix” supported by more detailed guides. The idea was that a checklist would help give a sense of accomplishment and manageability: if your documentation includes everything mentioned in the checklist, you’ve done a good job. The “matrix” component was an effort to guide people into a process for planning out how they’d write all their docs, rather than just asking them to sit down and start scribbling.
In the beta version</a>, I tweaked this design slightly. There’s still a checklist, front-and-center on the landing page</a>. But rather than show this checklist as a matrix, I lay out a structured approach in a “playbook”</a>, a suggested workflow of about 20 steps that aims to guide people from a (metaphorical) blank page to a filled-out checklist. As soon as I created my first checklist matrix mockup for presentation at the US-RSE'25</a> meeting last fall, I could tell that it just had too many boxes, and I never found a way to fix that fundamental issue. In comparison, I’m much happier with the playbook model. It provides a structure for tackling the checklist, but avoids overcomplicating the checklist itself, and I think the framing makes it clear that if you want to tackle the checklist in some other fashion, that’s more than fine.
On the website I deliver the playbook as an HTML slideshow</a>, following in the footsteps of some of my work on DASCH</a> (and, more broadly, a hobbyhorse I’ve been riding for more than a decade</a>). I’m getting more and more enthusiastic about HTML slideshows as a very useful form-factor for pedagogical, web-based documentation; I’m coming around to believing that they’re the best way to avoid the “wall of text” fatigue that arises so easily when reading lengthy material on a screen. I hope other people actually agree!
One thing that didn’t change from my initial plan is that in addition to the checklist and playbook I do indeed have a series of “in-depth guides”</a> offering advice about how to prepare different elements of the documentation checklist. Some of them aren’t really about writing; two elements on my checklist are citation information</a> and a licensing statement</a>, both of which are the kind of thing that may only take up a few sentences in one’s final documentation, but may require a great deal of prep work in order to be able to write those two sentences. These guides will probably need tweaking here and there but I’m very happy with how they’ve turned out so far, and I’d like to think that they might become generally useful resources going forward.
I certainly have ideas about how to make One Good Tutorial even better, but I can tell that it’s ready to be sent out into the world for beta-testing. So, here we are! I’ll be reaching out to some folks individually, but if you’re reading this and and you have some experience with documenting scientific software, I’d love some feedback. You can get in touch via the One Good Tutorial repository</a> on GitHub or by contacting me directly</a>.
Over the last few months of my fellowship I’ll be polishing the website and working to get the word out. I’m quite proud of how this project has come together so far and I hope you find it useful! Check out the One Good Tutorial materials at onegoodtutorial.org</a>.
The work described in this post was supported by a Better Scientific Software Fellowship</a>.

One Good Tutorial: The Plan

Fri, 29 Aug 2025 14:02:11 -0400

The past few posts have been about prep work for my BSSw project</a>: interviews</a> and a survey of tools</a>. After all this throat-clearing, I’m ready to sketch out the resource that I’m actually planning to create! 
I plan to call it One Good Tutorial. The target audience will be, of course, developers of small-to-medium scientific software projects. The centerpiece of the project will be a checklist: complete these actions, and you can sleep easy knowing that you’ve documented your project adequately.
One thing that I absolutely want to beat people over the head with is the point that there is more to documentation than API docs. This is the big idea behind Diátaxis</a>, of course, and I’m completely sold on it; and I also believe that it’s an idea that many scientific software developers need to be exposed to. I think this is such a big deal that, well, I let it drive the whole project branding. In particular, I believe that the most important thing that most projects lack is introductory, “getting started”-type material. So: if your docs have One Good Tutorial, you’ve done your job. If the only thing that people retain from being exposed to my project is those three words, I’ll be happy.
I also believe that many scientific software developers don’t feel confident about how they should approach documentation in general. This is, well, totally reasonable: technical writing and information architecture are whole fields of human endeavor, and we’re generally approaching them with no training or support. Realistically, that’s not going to change: the goal is not to train people to become expert technical writers. But I think it will make a real difference if we can help scientific software developers feel like they’re not quite so at sea. Hence the checklist format. I’m hopeful that a checklist will work well to provide both tangible instructions and a rewarding sense of clarity: “OK, I did everything they said I should — gold star for me!”
I also think that such a checklist will fill an unoccupied niche in this space. The Good Docs Project</a> provides templates for authoring specific documents, but doesn’t quite provide the holistic work plan that I think a checklist will offer. The Write the Docs Guide</a> has a lot of resources and guidance but, once again, doesn’t quite meet the needs of someone saying, “Just tell me what to do!”
Another nice aspect of the checklist format, I think, is that it leads to a natural structuring of the resource materials. The core artifact is, of course, the checklist itself, which I’d expect to deliver as both HTML and a PDF one-pager. Then, for each item on the checklist, there will be an associated webpage with deeper explanation, references, and examples. In certain cases this page might be quite short, but in other cases, it could get fairly extensive. Contrast this with “cookbook” or “recipe” format, which tend to be structured more like prose text — which means that the length keeps on increasing as you think of little details or clarifications to throw in. The recipe format also implies that the steps should be followed in strict order, whereas checklists allow for some level of skipping around. I think that’s a good thing in this case.
More specifically, I’m currently envisioning what you might call a “checklist matrix”. The checklist items will mostly correspond to important pieces of documentation that must exist: Tutorial (of course!), Citation Information, Installation Instructions, and so on. These are the rows of the checklist matrix.
But I’m also envisioning four columns corresponding to four phases that I will encourage people to work through: Plan, Draft, Assess, and Revise. The basic guidance would be to go through these phases in order:

Take some time to think about your plan for all of the different components of your documentation. Consider creating a Google Doc, or something similar, to hold notes about your plans.</li>
Actually draft the materials, and do the initial setup of whatever tools you’re going to need to get your docs published.</li>
Assess the complete first draft. Did the process of drafting reveal any problems that need to be fixed?</li>
Revise. Self-explanatory.</li> </ol>
I pointedly do not include a “publish” phase, because I think that encourages people to think of the docs as a one-time project: “I wrote them and published them, and now they’re done.” I think it’s important to approach the both the code and the docs as things that are never quite done, which to me means having a mindset oriented around “making releases”, rather than “publishing”.
Here’s my first draft of the rows for the checklist matrix:

Synopsis. A few sentences summarizing the software. Good to do first because it helps you keep the big picture in mind, and it’s likely to be copied around in READMEs, website landing pages, package descriptions, etc.</li>
Personas. I want to encourage people to take a few minutes to imagine user personas for their documentation: who’s going to be reading the docs? I hope this will be genuinely helpful for people as they’re thinking about docs … and I don’t hate the idea of sneaking in an idea that they might be able to apply much more broadly, too. A row like this one would have checkboxes for the Plan and Assess phases, but not Draft or Revise, since it doesn’t explicitly appear in the documentation product.</li>
Tutorial. This had better come early! I hope that suggesting that people sit down and plan their tutorial before actually putting (virtual) pen to (virtual) paper will help them think through bigger-picture issues: “oh, the user is going to need to have access to some 10-gigabyte data file, so I had better come up with a way to distribute it”. If people just plunge straight into writing the tutorial (perhaps after burning out developing the code), that’s the kind of problem that gets left unresolved.</li>
Installation Instructions. Just one of those things that you need to have.</li>
Citation Instructions. This is one of those sections that is unlikely to take up many words in published documentation, but I really want to make sure that people sit down and think about how they want to approach it. It would be beyond the scope of One Good Tutorial to tell people what approach to citation to adopt, but the supporting materials can point them to resources on the topic.</li>
License. This is similar to citation instructions: a lot of people just don’t even think about this issue.</li>
How to Contribute. Another item that I find is often overlooked. This is the sort of thing where I can offer people boilerplate language to use. This is where I would suggest that larger projects think about adopting a Code of Conduct too.</li>
API Reference. I want to make a point of putting this really far down in the list … but yeah, this is important for software.</li>
Other Reference Materials. I’m not quite sure how to name or describe this element, but in scientific software, there’s often some kind of theory underlying the software, and it’s really important to document it precisely. In many cases, the documentation here might basically consist of a reference to a formally-published journal article.</li>
Acknowledgments. Don’t forget to thank your funders! I will also ask people to acknowledge One Good Template if they have found it useful.</li>
Authoring Tools. Once someone has gone down the list and thought about all of the above materials, now it’s the time to think about: what tools are we going to need to create this documentation? For some projects, you could absolutely cover every item above in a single README.md</code> on GitHub; for others, you want to think about whether Sphinx (etc.) will suffice, or whether you might need to adopt a combination tools. The Draft phase is where you would actually start wiring these tools into your workflows such as CI.</li>
Release Processes. Once you’ve thought about what your docs are going to look like and what tools you’ll use to write them … how specifically are they going to make it out into the world? As mentioned above, here I want to encourage people to think of doc publication as an ongoing process, possibly one that’s integrated with the software release process.</li> </ul>Each of these items will have a corresponding article on the One Good Tutorial website, providing advice on how to approach the item in each of the four phases. In at least some cases, these will branch out into more specific how-to pages. For the Authoring Tools and Release Processes items, this is where I will provide specific tool recommendations and step-by-step tutorials on how to handle common scenarios (e.g., using Sphinx and ReadTheDocs to document a pure-Python package; depositing your software to Zenodo). There should be ample opportunity to refer people to existing resources like the Good Docs Project</a> templates. Time permitting, I could also see myself adding supporting “explainers” giving information about, say, the topic of software citation in general. It will probably also make sense to have a section that I would describe as “Extra Credit”. This would mostly be aimed at slightly larger projects, addressing topics like codes of conduct, organizing multiple tutorials, how-tos, social sites like StackExchange, and so on. There’s no shortage of material that could be written here, but I expect that these topics will be out of scope for most of the developers that I would want to visit One Good Tutorial. And in the end, I’m aiming to reach people rather than projects, so I’d rather be relevant to lots of people working on smaller efforts, even if the bulk of the documentation-reading and -writing that happens might be concentrated on a small number of high-profile pieces of software. I’m feeling pretty good about this plan, so I’ve gone ahead and registered onegoodtutorial.org</a>, set up a GitHub repo (pkgw/onegoodtutorial</a>), and wired up a static site built with Zola</a> and hosted via GitHub Pages</a>, with deployment automated using GitHub Actions. I am pretty sure that a basic static site generator will suffice for setting up the OGT website; if I run into limitations, it will be easy to rebuild it to use different infrastructure instead. The work described in this post was supported by a Better Scientific Software Fellowship</a>.
The State of the Doc Tools Thu, 28 Aug 2025 15:13:21 -0400 Last week I presented</a> some of my takeaways stemming from a small set of interviews with researchers about scientific software documentation. This week, I’m reporting out the results of a survey and review of tools for documenting research software: The State of The Doc Tools, 2025. An early, big-picture finding: as a result of both my interviews and my survey, I’ve concluded that it’s really important to construe the words “documentation” and “tools” expansively. That first word, “documentation”, might conjure up an image of a hefty spiral-bound manual. But that’s just one form-factor out of many possibilities. I think that more than ever, it should be emphasized that documentation can live in some surprising places. YouTube videos? Sure</a>! StackExchange Q&As? Indeed</a>. Discord chat groups? Empirically, yes</a>. Zines? Why not</a>? Custom-trained LLMs? I wouldn’t be surprised if they become standard for large projects, soon. That’s not to suggest that one ought to try to occupy all of these spaces — especially at the scale of a typical scientific software project! — but I think it’s really important to think very openly about where your documentation could live. And perhaps where it does live, in practice. Maybe a hefty book is right for your project, but maybe not. The corollary of this, however, is that it’s impossible to survey documentation tools in a truly comprehensive and systematic way. So, I won’t try to present an exhaustive list of software that one could use. But that’s not all. I also wrote that I think it’s important to construe the word “tools” expansively. Sure, there are things that everyone would agree fits that definition: Sphinx</a>, mdBook</a>. But what about Diátaxis</a>, the “four kinds of docs” paradigm that I’ve written about before</a>? If we think of a “tool” as “something that helps us accomplish a goal”, then I would say that Diátaxis absolutely fits that definition. And that is indeed what I’m saying! Last week</a> I observed that, despite valiant efforts to build one, there is not much of a “community of practice” for many of the people who are documenting scientific software. I believe that the lack of this community, while it definitely doesn’t do us any favors when it comes to tangible tools like Sphinx or mdBook, is even more damaging when it comes to intangible, intellectual tools like Diátaxis. The key difference is that intangible tools tend to remain nebulous, un-named, un-documented, and therefore more difficult to research on your own. I keep on coming back to the Diátaxis example, after all, because Daniele Procida has turned what could have been a somewhat vague mental model into a tangible, referenceable thing! (We see a bit of this now in how everyone falls over themselves to brand their security vulnerability discoveries</a>.) If we had stronger communities of practice, the intellectual tools would spread more easily. In the meantime, more people should follow Daniele’s lead. All that being said, even after doing a fair amount of digging I’ve been unable to unearth many other “intangible tools” to include in my survey. There is the Information Mapping</a> methodology, which might be useful, but I’m not going to find out because it’s a proprietary system (!) — you have to pay to be taught the model. For both practical and ethical reasons, I consider this to be a total non-starter. The kinds of intellectual tools that I’m looking for would, for the most part, probably be found in the field known as Information Architecture</a> (IA). There are techniques associated with the IA field like card sorting</a> and tree testing</a> that could be useful tools in the software documentation toolbox to sit alongside Diátaxis. I’ve found the website of an outfit called the Nielsen Norman Group</a> to be surprisingly useful in learning about some of these concepts. While NNG is your typical corporate consultancy trying to make a buck off of you, I’ve found their materials to be much more useful than the usual content-marketing drivel that you find on such sites. I’ve also bought a few IA books (including the aptly-named Information Architecture</a>) and will see if any of them seem worth recommending to non-specialists. Turning back towards the traditionally-recognized software documentation tools, we can return to Sphinx</a> as a starting point. One direction to go is to consider comparable tools aimed at other programming systems (granting that Sphinx is not actually Python-specific), which of course are numerous: tsdoc</a> for TypeScript, rustdoc</a> for Rust, Doxygen</a>, godoc</a>, Swagger</a> for web APIs, rdoc</a>, and on and on and on and on. It’s understandable that most modern programming languages deliver integrated documentation systems; each language has its own distinctive semantics that need to be captured in its API documentation framework. It’s also understandable that developers may be naturally inclined to try to author all of their documentation using these tools. But it’s worth pointing out explicitly that there’s no particular reason that a tool designed to document APIs in a certain language will be very good, or even adequate, for authoring general-purpose, non-API documentation. A very good reason to try to stick with language-specific documentation tools, though, is that modern ones can pair with public services that allow you to host documentation online for free. ReadTheDocs</a> is the standard for Python, docs.rs</a> for Rust, and in a certain sense, CTAN</a> for LaTeX are examples. If there’s one thing I’ve come to appreciate over the years, it’s that to first order nobody wants to host their own content, and in my experience services like these have been transformative for how scientific software is documented. That is, the universe of documentation tools to consider includes not just authoring tools, but also publishing platforms. Continuing along that thread, there are huge numbers</a> of static site generators out there that can (or could) be used to generate software documentation websites, as well as “* Pages” hosting services to host such sites for free. A random sub-sample from the former group: Jekyll</a>, Hugo</a>, Gatsby</a>; I’m partial to Zola</a> and have used it for docs; Docusaurus</a>, MkDocs</a>; mdBook</a>, GitBook</a>. In the latter group: GitHub Pages</a>, GitLab Pages</a>, Netlify</a>; ReadTheDocs</a> can effectively act as a static pages host; and all of the cloud providers make this drop-dead easy. This is a completely saturated market. A market that’s a bit less saturated, somewhat to my surprise, is the one for wikis. This is probably because wiki software needs to implement both authoring and publishing (i.e., hosting) features, unlike the static-site case where there’s a clean separation between the two halves. But more and more I’ve come to feel that the basic wiki paradigm is a strong one — it works pretty well for one of the top ten sites on the internet</a> — and that it would be a great fit for a lot of use cases in the software-documentation. But for some reason wiki software tends to have a 90’s feel, and a 90’s look as well. I’m becoming more and more convinced that there’s a lot of untapped opportunity in this space. Anyway, some of the main wiki tools are: MediaWiki</a>, GitHub’s wikis</a>, DokuWiki</a>, MoinMoin</a>; you could put Confluence</a> into this box too. There are other platforms that, integrate authoring and publishing like wikis, but have modernized WYSIWYG styles. Curvenote</a> (a commercial product) is specifically aimed at scientific authoring. PubPub</a> is a unique open-source platform aimed at a more general academic audience (combining Google-Docs-like collaborative editing with features like DOI minting and citations), but sadly the project’s very existence is under threat due to the withdrawal of a major funder. PubPub is the only substantial noncommercial player in this space that I’m aware of, so I really hope something the worst doesn’t come to pass. (In case it’s not obvious, I’m focusing almost exclusively on open-source and noncommercial tools here; beyond generalized academic cheapness, I believe that there are profound reasons to prefer them in this domain.) I’m a bit suprised that Overleaf</a>, the online collaborative LaTeX editor, hasn’t gotten into the publishing business, but as far as I can tell they haven’t yet. If you want to go a little bit farther, you can think of mainstream social media platforms as being on the same continuum. It’s not incorrect to describe YouTube</a> as a platform for creating and publishing content, after all, and one can imagine scientific software projects where it would not be an unreasonable place to host (nontextual) documentation. Likewise, the bulk of the documentation regarding some software projects probably lives on StackExchange</a> sites, in the form of answers to user questions. This line of thought brings us to publicly-visible “forum”-type systems (Discourse</a>, phpBB</a>) and more-synchronous “chat”-type platforms (Discord</a>, Gitter</a>, Zulip</a>, Slack</a>) which are usually less- or non-public. These platforms are not generally what people think of when they think of “documentation”, but I reiterate that it can very much happen that they end up hosting a significant or even dominant portion of the recorded information about a software project. Finally, we can narrow our focus to the tools used to produce individual documents. In the context of our touchstone Sphinx, this takes us to the underlying markup options like reStructuredText</a> and the hugely popular family of Markdown</a> syntaxes (less well-defined than you would hope</a>), especially the relatively new entrant MyST</a> which emphasizes technical applications. My perennial favorite TeX/LaTeX</a> is relevant, but for the vast majority of users it only targets PDF output, and you’ll note that virtually everything that I’ve discussed revolves around HTML and the web browser. (To me, this is the fundamental issue holding TeX back in the 21st century.) You can think of the Jupyter notebook</a> as a document file format (one that happens to come with a standard WYSIWYG editor), in which case nbconvert</a> joins pandoc</a> in the category of tools that connect the low-level document file formats to higher-level systems like Sphinx and mkdocs. Whew! I could keep going, too, but I think I’ve managed to touch on the major categories of tools that go into producing software documentation. Synthesizing all of the above, I think it might be not be unreasonable to talk about documentation as being produced within documentation systems. Here I’m referring only to the technical implementation, not conceptual tools like Diátaxis. I’ll claim that these technical systems include four kinds of technologies: Low-level file formats of individual documents</li> Tools for authoring individual documents</li> Tools for assembling documents into structured collections</li> Tools for publishing (i.e., hosting) such collections</li> </ol> Some tools blur the boundaries between these layers, and you could probably write a whole PhD thesis on the definition of the word “document”, but I’m going to claim that if you look at anything that you can call “documentation” with a straight face, you’ll be able to meaningfully isolate the technologies that are used for each of these four layers. The work described in this post was supported by a Better Scientific Software Fellowship</a>. The State of the Docs Fri, 22 Aug 2025 13:04:45 -0400 As part of my BSSw Fellowship</a> project on scientific software documentation, I’ve conducted free-form interviews with some of my colleagues to try to learn a little about how working scientists are approaching research software documentation in the year 2025. In this post I’ll report my findings. But first, some disclaimers. I’m describing an extremely modest, qualitative undertaking. While I very much enjoyed my conversations, and so have plans to interview a few more people (reach out if you’d like to volunteer!), at the moment my sample size is n = 4. Even if that sample size were a lot bigger, I’m not doing structured interviews here (nor am I trained to do so), so this is far from rigorous in any sense of the word. I’ve also tried to pursue a group of interviewees that’s relatively diverse, but … again, we’re talking about four people here. Three of them are astronomers, and of course the set of people willing to sit down and talk to me is a biased subsample of the universe of people working with research software. All this being said, I suspect that the patterns I’m noticing are likely to generalize. If I had to choose one word to summarize how my interviewees felt about scientific software documentation in general, I’d go with dissatisfaction. I think it's fair to say that everyone I talked to has a real appreciation for good documentation; but no small amount of that appreciation stems from its rarity. Good documentation is hard to find. Part of the reason for that is intrinsic: it’s hard to write well! But, based on my interviews, I’m not alone in feeling that there are a lot of extrinsic factors holding things back, especially authoring tools that are limited and limiting. In that context, I was a little bit surprised that none of my interviewees complained about documentation being unappreciated. A perennial — and completely valid — complaint from scientific software developers is that they don’t get the recognition that they deserve from their peers. I had expected that at least one or two people would report that they were discouraged from spending time on docs because it seemed like the results weren’t being valued. But no one offered that sentiment, although I didn’t make a point of trying to elicit it either. Nor did anyone bring up the challenges of making documentation work legible to the traditional academic reward structures (e.g., making docs citeable), which is another preoccupation when it comes to research code. It could be that, given a baseline expectation that time spent on research software is going to be underappreciated in general, a lack of appreciation for time spent on software docs goes without saying. In a similar vein, only one interviewee specifically mentioned the connection between funding and software documentation, or the lack thereof. I’m confident that the potential impact of funding is obvious to everyone involved; it’s just that the prospects for any kind of systematic financial support for software documentation feel grim at best, at the moment. I can imagine the contours of a pitch to change this: fundamentally, documentation is education, and education deserves investment, right? Well, lately there’s less agreement on that point than I’d like. But I’d be happy to argue that most students would learn not just as well but actually more effectively from great software documentation than from a great textbook: it’s hard to beat learning by doing. Anyway, maybe one day there’ll be somebody to listen to this pitch, but I’m not holding my breath. The lack of systematic support — both financial and less tangible kinds — surely has much to do with another theme that I noticed: a lack of a community of practice</a> around research software documentation. It feels (emphasis on the feels) like everybody’s out there on their own, figuring things out independently. While some of my interviewees mentioned learning how to create better documentation from looking at other docs, no one gave much indication that they felt that they had learned much from other people. This is another result that can be filed under “Absolutely Zero Surprise Involved”. But I do want to emphasize that there are people out there actively trying to create exactly these communities. The two groups that come to mind immediately are Write The Docs</a> and The Good Docs Project</a>, the latter of which I only learned about recently. I’m sure there are more, and everything I’ve seen suggest that the people in these organizations are doing exactly what they ought to — it’s just that this kind of community-building is slow, tedious, difficult work. It’s also worth pointing out that there is a much bigger world out there once you start considering the field of technical writing in general, as opposed to the narrower group of scientists-doing-software-docs-amateurishly. (On the other hand, I was going to link to the Society for Technical Communication as an example, but I see that they closed their doors earlier this year</a>.) Getting a bit more pragmatic, one high-level idea that I took away from my interviews is that there are basically two levels of documentation. At the lower level, we have the things that we can think of as, approximately, individual documents: a single tutorial, or the API reference for a single Python package. At the higher level, we have document collections: a whole suite of tutorials and API references, design docs, and so on. At both levels, we face challenges relating to design, organization, and tooling, but the details of those challenges are actually quite different. Smaller projects are basically operating only at the lower level, but for people working in larger projects, it seems that attention quickly migrates to primarily focus on the high-level issues, rather than the details of individual documents. For my BSSw project, I plan to primarily address the lower level — preparing individual documents — but I came out of my interviews feeling that it’ll be helpful to draw a line between the two levels (even if it is, inevitably, fuzzy) and offer some guidance about how to tackle the higher-level issues. Another theme from my interviews was interest in experimenting with different documentation “form factors”. I think that Jupyter notebooks came up in all of my discussions, and everyone was pretty much on the same page in feeling that they can be a very good vehicle for certain kinds of software documentation. I brought up videos in a few interviews, because I’ve been struck by the seeming trend in industry to produce video documentation for everything, even cases where it feels like a few paragraphs of text would be more than sufficient. I still don’t quite understand where that’s coming from — and none of my interviewees seemed to see any special appeal either. There are cases where video documentation can be very helpful (as I’ve experienced with WorldWide Telescope</a>) but, given that it takes a lot of work to produce, my interviewees’ conception of “documentation” still centers on text. One interviewee was sympathetic to an idea that’s been bouncing around in my head: perhaps we should be making a lot more use of slide decks for documentation. There’s no need to belabor the ways in which they can go wrong, but at their best, slide decks can break down material and interleave text and graphics in a way that seems a lot more digestible than a linear document. And, as I’ve harped on for, gosh, more than a decade</a>, you can make nice HTML decks that can be viewed without the need to “context switch” into a dedicated app. I tried this approach in the DASCH documentation</a> and am personally pretty happy with the results, although the scale of deployment is small enough that I don’t have any firm feedback about what anyone else thinks. One of the challenges for this idea is that authoring these decks is a bit tricky, although maybe most people would be satisfied with links to Google Slides or something along those lines. Attention to authorship barriers — how hard or easy is it to just sit down and write something? — is looming large in my thinking as a result of these interviews. One interviewee was adamant: write docs using whatever tool you have at hand; the best doc is one that actually exists. Others were focused on getting the right tooling into place: setting up CI to run doctests (using a tool like Sybil</a>) ensuring that they never break, or to automatically execute Jupyter notebooks and integrate them into a Sphinx tree. Both of these attitudes have their appeal, but there’s definitely a tension between them. I’m not quite sure how to navigate this tension in the resource that I’ll be creating. My personal bias ought to be pretty clear: whatever the opposite of quick-and-dirty is, that’s usually me. Turns out, though, that most people aren’t like me. I’d like to think that my resource can offer people some cookbook recipes to help them get the benefits of some of the more complex tools (it’s good to verify that your example code actually runs!) without getting mired in a slog of Sphinx configuration, but I’ll have to be careful not to go overboard. The work described in this post was supported by a Better Scientific Software Fellowship</a>. MPC and the Rubin First Look Wed, 02 Jul 2025 13:25:47 -0400 At long last, the first data from the Vera C. Rubin Observatory</a> are starting to become public! Here at the Minor Planet Center we were not only watching last week’s Rubin “First Look”</a> event — we had some work to do too. Asteroid-hunting and planetary defense</a> have always been a major piece of Rubin’s science case. When it hits full steam, Rubin will increase the rate of observations coming into the MPC by a factor of around five, and at this point years of effort have gone into getting the MPC ready for Rubin’s data stream, as I’ve mentioned a few times</a> already</a>. With Rubin getting closer to full operations, the MPC and Rubin teams have been working together closely to build the actual systems that will send Rubin data to MPC and process them. We collectively agreed that as part of last Monday’s launch event, the MPC would accept a first official Rubin data delivery and process it for distribution through our standard, public-facing interfaces. Quite a big milestone for this long-lasting project! While the Rubin team submitted their data in the same way that everyone else does, we processed the measurements using a set of next-generation pipelines that we’ve been developing as a major part of the broader effort — the first time that we’ve done so as part of production operations. This wasn’t strictly necessary, in a certain sense — the legacy pipelines that churn away every day could, in principle, have handled the data. The old and new systems are functionally equivalent, at the moment, and the data volume of this one submission wouldn't have been enough to tip our systems over. But the whole point of building the new pipeline is that once we start getting Rubin data every night — which will start happening in a matter of weeks! — the legacy system simply won’t be able to keep up. The time to get the new code up and running is now. Now, the dirty secret, such as it is, is that we already processed Monday’s submission from Rubin dozens of times. We have a “sandbox” system that’s been hosting the new pipelines and we’ve been testing its results assiduously. So we knew exactly what we were going to be getting, well in advance. But, that being said, there’s always a difference between testing and actual production deployment, so Monday was genuinely a big day for us. How did it go? Pretty well! We did uncover some issues that needed working through, but I would generally characterize them as ones that I don’t feel too bad about — issues occurring in the gaps that we knew we wouldn’t realistically be able to test well in advance. Probably the most glaring issue was that the provisional designations</a> generated by the new code were associated with the wrong dates. That wasn’t ideal, but it was a one-off mistake, rather than an indicator of a flawed design. MPC developer Brian Burt</a> did a fantastic job squashing this and other issues last week, allowing us to process another batch of Rubin data on Friday with nearly ideal results. I won’t run down all of the issues that we ran into, but there was a clear theme, alluded to above: the problems occurred in places where there were differences between our test environment and the actual production environment. (Twelve-factor</a> wins again.) Or, in some areas like the issuing of provisional designations, at the moment we simply don’t have a way to test such code in an end-to-end fashion in a non-production setting. No surprise that that’s an area where problems surfaced. Fortunately — in a certain sense — this theme came as no surprise to us. We’re well aware of the limitations in our software testing capabilities, and have been working steadily to address them. We have what we need when it comes to unit testing, but integration tests are another story: we simply don’t have non-production versions of various subsystems needed to test end-to-end workflows. I was not at all surprised to discover this when I arrived at the MPC. In my experience, the idea of having parallel test and prod deployments is a great example of something that’s near-universal in industry, but that a surprising number of self-taught academic software developers aren’t used to. (Another example: semantic versioning</a> and the concept of API breakage.) Historically, MPC certainly had nothing of the sort. But that’s something I intend to make sure we fix — a lot of my preoccupation with code deployability and topics like release automation</a> is precisely because these practices help you ensure that you can construct multiple identical deployment environments, which can then be used to support complex testing and prototyping. Since MPC’s technology systems are both complex and legacy-filled, it will take a long time to complete the evolution, but I’ve found that it’s always worth the effort. CASA 6 is Now in Conda-forge Tue, 29 Apr 2025 10:08:38 -0400 I’m pleased to report that CASA 6</a> is now available in conda-forge</a>! CASA is a software suite for processing radio astronomy data from telescopes such as the Very Large Array</a>, ALMA</a>, and more. The availability includes the casatools</code> and casatasks</code> Python packages but not the full suite of CASA end-user applications. Just run: $ conda install casatasks</code></pre> or the equivalent command in a Python environment that has conda-forge enabled</a>. Somewhat terrifyingly, I’ve been working on packaging CASA for almost exactly a decade</a>. The origin of this is that like many complex data reduction packages of its time, CASA was (and still is) distributed by NRAO</a> as a large, self-contained software environment: the latest CASA installer for Linux is just shy of a gigabyte in size, including not just data files but also all of the support libraries needed to ensure that the binaries can run on as many systems as possible. The issue with CASA was that it was also trying to embrace Python as a scripting language. In NRAO’s monolithic distribution model, this meant embedding an entire freestanding Python interpreter in the CASA distribution. This might seem like a minor packaging choice, but my claim is that it hugely affects how you can work with the software. A key advantage to a scripting language like Python is that it allows you to bring together a huge range of codebases in one “place:” you can write a program that glues together the netlib</a> numerical libraries via Scipy</a>, with GUI toolkits like GTK</a> using PyGObject</a>, and data I/O packages like HDF5</a> through h5py</a>. When a package like CASA bundles its own interpreter, if you want access to these kinds of libraries you can’t just reuse whatever software collection you’ve painstakingly set up over the years — you have to install everything from scratch into its environment. In this model, CASA isn’t a tool to add to your toolbox — it’s a quarantine zone that can only be entered or exited through an airlock. Even worse, back then CASA’s Python installation was missing key development files, so that many packages with binary components couldn’t be installed into its environment at all! At least, not unless you were willing and able to devise some extreme hacks to fill in the needed files. Of course, NRAO wasn’t distributing CASA as a monolith out of spite: for a long time, that was the only realistic way to deliver a large application (or suite of applications) with complex, specialized dependencies. But in 2012 (around the time of CASA 3.3</a>) Anaconda first released the conda package manager</a>. I’ve mentioned before that in my view much of conda’s design was more or less evolutionary</a>, but that’s not to downplay its impact — in my view it has truly transformed the way that we can deliver scientific software to users. In particular, conda and conda-forge</a> have combined to create an ecosystem where it can be amazingly straightforward to install complex dependencies into arbitrary Python environments. Back in 2015, that was what I wanted for CASA. So I started slogging through its obscure and out-of-date dependencies and developed conda recipes for the whole stack</a>, starting with version 4.4. And lo, it was good. I did discover that I had to write a whole bunch of code</a> to make a bunch of CASA’s functionality meaningfully usable from Python. It turned out that despite having its outermost layers written in Python and claims of “scriptability”, much of the CASA simply wasn’t usable like a regular Python package. In 2017 (CASA 4.7, Python 3.6), it was clear that it was time to really shift to Python 3. But it was also clear that CASA wasn’t going to be supporting Python 3 for a long time yet, and that there wasn't anything that individuals outside of NRAO could do about that. So I added Python 3 support to CASA myself</a>, which was … painful. But once again, I found it worthwhile to be able to actually have CASA be a first-class member of my general software toolkit. Around 2019, CASA 6</a> was coming out, which both added support for Python 3 and started the process of making CASA’s architecture more Python-native. But I had started spending a lot less time thinking about astrophysics</a>, so I didn’t spend much time exploring it. I took an initial look at updating my recipes for CASA 6, saw that things seemed about as challenging as in the CASA 5.x series, and decided not to worry about it for a while. Now, after a long hiatus, I’ve had some reason to take another look at updating my CASA conda packages. Finally, finally, the upstream source code is in pretty good shape! I was able to put together conda recipes for casacpp</a>, casatools</a>, and casatasks</a> over the course of just a few days. Instead of thousands of lines of patches, I only needed a handful. And pretty much all of the old, outdated dependencies needed by CASA 4/5 are now gone. The combination of all of these factors made it feasible for me to get these recipes integrated into conda-forge, rather than building the packages myself. This will offer massive maintainability gains going forward, thanks both to conda-forge’s impressive infrastructure and a much more realistic possibility for other people to help keep the packages up-to-date. Huzzah! If you want to install CASA using these packages — or, really, create and manage any kind of customized software environments — I highly recommend using Pixi</a>. Run pixi add casatasks</code> and you should be good to go. But you can use these packages to install CASA into any conda-backed environment just by activating conda-forge</a> and running the analogous installation step. As for why I’ve returned to CASA … I would say “more soon,” but that’s unlikely. My wife is due to give birth to our first child within the week so I don’t expect to be posting much for a while! I won’t be able to take as much of a formal leave as I’d like because I’m within my first year of Smithsonian employment and hence not eligible for FMLA</a>, but I have a sneaking suspicion that my hands will be pretty full for the foreseeable future. “Generic” Artifacts in GitHub Packages Thu, 03 Apr 2025 11:38:56 -0400 Service blogging today! For a while I’ve been pondering if it would be possible to use the GitHub Packages</a> service to host “generic” files: namely, arbitrary binary artifacts that aren’t necessarily NPM packages, Docker images, etc. Motivated by some of the my current MPC</a> projects, I sat down this week to look into the topic more deeply than I have before. Lo and behold, you can do this! And it isn’t even (that big of) a hack. Warning: I realized that I wrote this blog like some ridiculous internet casserole recipe. Skip down to the code blocks at the end if you just want to see what to do. Back when I was a lad, installing software was an adventure: for every program you needed, you dug up its website, found the Downloads page, pulled whatever file(s) the authors provided, and figured out how to actually get the damn thing installed. (OK, well, actually, I remember the days of installing software from stacks of floppy disks, but we're not going back that far.) From the very earliest days of the internet, though, people saw the value of pulling files into shared repositories: CPAN</a> and CTAN</a> were among the first; then we had Linux distributions that packaged up and hosted amazingly wide-ranging collections of software. But I feel like it took a while for people to appreciate just how valuable these systems could be; I remember being struck by the remarkably tight integration between the npm</code> tool and npmjs.com</a> when they launched, which was 2010. Nowadays, you would be foolish to launch a new language or framework without some kind of central package registry. But we're actually seeing a trend towards de-agglomeration as ecosystems get so large and complex that you start running into problems if you're limited to a single, global package namespace. For instance, while we started with a single original Docker Hub</a> for hosting Docker container images, we now live in a world where you can spin up your own organizational registry using infrastructure provided by Amazon</a>, Azure</a>, or Google</a>, not to mention many other options. You see the same pattern for NPM, Cargo, Conda, and other major packaging ecosystems as well. (As a side note, this emergent flexibility is a testament to the brilliant simplicity of the Internet’s architecture! None of this would be possible without the URL. Good job, team.) In 2019, GitHub joined the fray with its own package hosting infrastructure: GitHub Packages</a> (GHP). While a lot of people might only be familiar with GHP through the GitHub Container Registry</a>, the subset of the service that deals specifically with Docker containers, it also supports NPM, RubyGems, Maven, Gradle, and NuGet. You can see how all of these systems might have a lot in common under the hood: they're all basically dealing with versioned sets of binary artifacts, and you can imagine building a common infrastructure for naming, hosting, access control, and more. That’s cool. But. What if I’d like to leverage the GHP infrastructure to manage a binary artifact that isn’t a Docker image, an NPM package, or any of those other things? A “generic” package, if you will — some kind of file whose contents could be anything? Obviously, if all else failed, you could embed your file in one of the schemes that GHP does support. You could write a Dockerfile that constructed a Docker image containing your file, and then you could fetch the image and extract the file. It’s not pretty, but it works — it’s an approach that I’ve used myself more than once. You could also do the same with NPM’s tooling, or probably any of the other packaging systems supported by GHP. Can we do better? Thankfully, we can. The short story is that nowadays you can use the GitHub Container Registry to manage generic packages in a pretty clean way. I’m not familiar with the detailed history, but as best I can gather, the Open Container Initiative</a> has driven the development of standards and tools to allow container registries to handle arbitrary file formats, and a side benefit is that we can (ab)use that support to leverage these registries even if our binaries don’t correspond to what we would normally think of as “container images”. In particular, there’s a tool called oras</code></a> that can talk to GHCR in a “generic“ way rather than a “Docker-specific” way. (It seems that ORAS stands for OCI Registry As Storage, based on the title of its webpage.) With this tool, it’s quite straightforward to deal with generic packages. Specifically, if you’re like me and you’d like to publish a generic package to GHCR in a GitHub Actions</a> workflow, all you need is the following: - uses: oras-project/setup-oras@v1 with: version: 1.2.2 # ... create `myfile.zip` somehow # $SLUG is your package slug, e.g. `pkgw/my-generic-package` # $TAG is the version tag, e.g. `latest` - name: Push package run: | echo ${{ secrets.GITHUB_TOKEN }} |oras login --username ${{ github.repository_owner }} --password-stdin ghcr.io oras push ghcr.io/$SLUG:$TAG --artifact-type application/vnd.pkgw myfile.zip</code></pre> It’s basically the same thing as pushing with the docker</code> CLI, except the artifact data come from a file on disk, and you need to specify an associated "media type". If you need your artifact to be consumable by third-party systems (say, Docker), you’re going to need to set up a variety of other metadata too. But if all you care about is pushing and pulling bytes, you can skip that, make up a meaningless media type, and call it a day. To retrieve your package later, it's exactly what you would hope: # this will create `myfile.zip`: oras pull ghcr.io/$SLUG:$TAG</code></pre> Boom, done! Readers experienced with GitHub will note that all of this might seem a bit redundant with GitHub Releases</a>, which you can also use to distribute versioned binary artifacts associated with your repository. That’s not at all off-base. As someone with plenty of experience automating the creation of GitHub releases</a>, though, I have to say that the GHCR approach feels a lot more lightweight. You don’t have to make up release notes, and you can just push a file instead of having to make API calls to declare a release and then attach artifacts to it. I also suspect that GHCR offers more fine-grained access control settings. For my MPC needs, I was willing to use the Releases system if it felt necessary, but I’m much happier to be able to use GHCR and oras</code></a> instead. The MPC is Hiring Fri, 21 Mar 2025 10:52:42 -0400 The MPC is hiring! We have two positions currently open — one Astronomer</a> and one IT Specialist</a> (i.e., software developer). Both applications close on March 31st. You should check out the links above for the formal descriptions of the two roles. Less formally, I’d say that there’s a wide variety of work and science that happens at the MPC and there are many different ways that a well-qualified person could contribute to the MPC’s success. I personally came into the MPC with essentially zero experience in minor-planet science (and it’s only been a few months</a> so that’s absolutely still the case) but that hasn’t hampered my ability to get involved, or my level of interest in what I’m doing! The job ads have to be a bit more neutral, but I’ll take the opportunity to try to sell the MPC a bit. Most narrowly, this is an important and exciting time for the MPC: with Rubin/LSST coming online soon</a>, the there’s about to be a boom in minor-planet science, as well as a five-fold increase in the rate of data coming into the MPC. Just as we start getting used to The Era of Rubin, NASA’s NEO Surveyor</a> will launch and ramp things up even more. All of this will mean that MPC will have a lot of work to do, but our impact and visibility will be higher too. Of course, in these (cough) “interesting” times it’s hard to feel like you can be at all sure what’s going to happen in a few months, let alone a few years. But for what it’s worth, the funding for MPC (and a lot of that for Rubin and NEO Surveyor) comes from NASA’s planetary defense program</a>; that is, the people who are worrying about city-destroying asteroids. Any scientist who tells you that their funding is totally secure is either delusional or lying, but I feel a lot better about MPC’s situation than some of the alternatives I can think of. Looking beyond the MPC’s immediate future, I think there’s a ton of opportunity for the MPC to make a much bigger impact on both science, and society more broadly. As I wrote before</a>, I see MPC as a perfect harbinger of where 21st-century science is heading: more and more, your ability to accomplish anything is going to depend critically on your ability to execute technology projects, above all software projects. Historically, MPC hasn’t done a great job of this, but my goal is to complete the task of turning that around. I’d love for MPC to be seen as a leader both scientifically and technically, and my sincere belief is that if you can excel on “both sides of the ball,” you’re going to be wildly more successful than if you’re only strong on one side. If you’re someone that feels the same way, please consider joining us! Fun with Databases Wed, 05 Mar 2025 13:35:03 -0500 Yikes, it’s been hard to keep up a weekly posting schedule lately. I’m hoping to ramp that back up, though — I’ve just been occupied the past couple of weeks with some nitty-gritty database performance work. A couple of weeks ago we migrated the MPC’s primary production database from physical hardware to a virtual machine running our “Borg” cluster, which runs the XCP-ng</a>/Xen Orchestra</a> virtualization stack. This database runs on PostgreSQL</a>, is about 2 terabytes in size, and really constitutes the beating heart of the MPC: it stores all of our core data assets, such as orbits, designations, and about 500 million astrometric observations. All of MPC’s operational systems are querying and modifying the database non-stop, 24/7. As one might imagine, you want to do such a migration carefully. Indeed, there were several months of planning and rehearsals leading up to the switchover. It went completely smoothly, thanks in no small part to all of the preparatory work. We’re now running the latest version PostgreSQL on a VM backed by a Powervault</a> storage array with automated snapshots, hot migrations between physical hosts, and all sorts of other nice resilience features. We did discover, however, that some aspects of the database performance weren’t living up to what we expected. In particular, there was a lot of variability: the same query would sometimes run quickly, and other times run orders of magnitude (plural!) slower. It wasn’t a crippling issue, but definitely something to address. And that’s what’s been keeping me busy since then. Pretty early on, we discovered that the variable performance had something to do with the particular way in which the VM was connecting to its Powervault storage. You can register Powervault volumes with the Xen system and then use them as “storage repositories” for virtual disks, which lets you manage and migrate them all through the Xen interface. But something about the Xen layer seemed to be responsible for the uneven performance — because you can also configure a VM to connect directly to the Powervault using iSCSI</a>, and doing so gave much more consistent results. Part of the reason might be that Xen disks currently are limited to a maximum size of 2 TB, so that we had to use LVM</a> in the database VM to create a sufficiently large virtual disk. On the other hand, while I’m sure this didn’t help performance, it’s hard for me to see how it would induce the variability that we were seeing. (For what it’s worth, that 2 TB limit is going away</a>.) Migrating from the Xen + LVM approach to the direct mount would require us to dump and reload the whole database, so we didn’t want to make that change unless we were convinced that it would help things. So, we spent a while running tests and trying to understand the performance characteristics of the stack, and then planning out the changeover once we were convinced that it was worth trying. Yesterday we made the change, and things have been a lot smoother since. The performance that we’re getting is still noticeably less than bare metal, but the benefits of using the virtualized system are extremely real. In particular I’m really excited that we have the option to “hot migrate” the database from one physical machine to another, which means that we can do hardware maintenance with zero downtime of the actual database service. But, if we really need to, we can sacrifice that and get better performance by adopting a technology called SR-IOV</a>, which basically allows the VM to access the physical host’s network cards in a lower-level, but still virtualized, way. In our low-level tests SR-IOV gave a ~50% performance boost for some workloads, which apparently stems from the fact iSCSI often uses a lot of little packets that require a lot of interrupts to service. For the time being, we’d rather keep the ability to hot migrate, but the SR-IOV option is there in our back pocket if we need it.