Eli Grubb · Notes

TIL: Testing Secure Zeroization in Zig with Custom Memory Allocators

Eli Grubb — Wed, 20 Aug 2025 00:00:00 GMT

or, How Zig’s explicit memory management makes it easy to test your sanitization guarantee

While spending the summer learning Zig I have come to love and appreciate one of the language’s core philosophies: what you see is what you get.

In Zig, there’s no operator overloading and memory allocations are always explicit. Simply reading the code should reveal exactly what your software is instructing the computer to do. If a library allocates memory, the idiomatic approach is to accept a memory allocator as an input, so callers can pick the memory management strategy that matches the end use-case.

There are exceptions,¹ and other languages have reasons they allow such abstractions, but I’ve enjoyed working in this style. It aligns well with my current interests and experience. When I return to other, more “sugary”, languages, using Zig has improved my feel for where the “hidden” allocations and functionality are.

An idea begins to form

In particular, this is useful for testing that your custom memory management strategies are, actually, working. My first “real” Zig project, the Zecrecy library, zeros out sensitive data before freeing the memory to mitigate the impact of Heartbleed-style vulnerabilities.

Because Zig is explicit and its docs are still maturing, you are reading the standard library’s source code by the time you build much more than “hello world”.² I’d read the code for the std.mem.Allocator interface, along with several of the allocator implementations in the standard library, so I felt confident enough to try it when the idea popped into my head: a custom allocator that panics if asked to free non-zero memory.

Of course, this is not a novel idea. I learned it from the Zig Standard Library’s testing Allocator, a nifty implementation that takes advantage of Zig’s memory management paradigm. In testing, use std.testing.allocator when a call requests a memory allocator. The test will panic if the allocator detects any memory leaks during testing.³

To improve Zecrecy’s testing quality, I wrote ZerosOnlyAllocator. It’s pretty simple, it wraps a child allocator to handle the actual allocations. This has a couple benefits:

I don’t have to write my own allocation logic just for a bit of testing, and
If the child allocator is std.testing.allocator, then I get memory-leak detection in the same test run!

The core “functionality” lies in the free() function:

/// Panics if the memory is not zeroed before freeing.
fn free(
    ctx: *anyopaque, 
    buf: []u8, 
    alignment: mem.Alignment, 
    ret_addr: usize
) void {
    const self: *ZerosOnlyAllocator = @ptrCast(@alignCast(ctx));
    for (buf) |byte| {
        if (byte != 0) @panic("non-zero byte freed");
    }
    self.child_allocator.rawFree(buf, alignment, ret_addr);
}

I love this kind of thing because it only gains value the more tests I add. Test setup is only three lines:

const ZerosOnlyAllocator = @import("testing/ZerosOnlyAllocator.zig");
var zeros_only_allocator: ZerosOnlyAllocator = .init(std.testing.allocator);
const allocator = zeros_only_allocator.allocator();

Now, every single test automatically verifies our core security guarantee with zero additional effort.

Debugging detour

Except! The first time I ran unit tests using the shiny, new, custom allocator, every single test panicked:

~/zecrecy> zig test src/secret.zig
thread 3381248 panic: non-zero byte freed
~/zecrecy/src/testing/ZerosOnlyAllocator.zig:59:24: 0x104dbc753 in free (test)
        if (byte != 0) @panic("non-zero byte freed");
                       ^
~/.cache/zig/p/N-V-__8AAPWKhxNMNK6YniIioDpRryBzI7DMp0hJB4rExlGU/lib/std/mem/Allocator.zig:147:25: 0x10236238f in free__anon_5332 (test)
    return a.vtable.free(a.ptr, memory, alignment, ret_addr);
                        ^
~/zecrecy/src/secret.zig:120:34: 0x10235e12b in deinit (test)
            secret.allocator.free(secret.data);

Turns out, my assumption that reading the standard library once makes one an expert was, unsurprisingly, wrong. The custom allocator’s logic was correct, but the standard library contained some unexpected functionality that broke my assumptions:

/// From `std.mem.Allocator`:
///
/// Free an array allocated with `alloc`.
/// If memory has length 0, free is a no-op.
/// To free a single item, see `destroy`.
pub fn free(self: Allocator, memory: anytype) void {
    const Slice = @typeInfo(@TypeOf(memory)).pointer;
    const bytes = mem.sliceAsBytes(memory);
    const bytes_len = bytes.len + 
                      if (Slice.sentinel() != null) @sizeOf(Slice.child) else 0;
    if (bytes_len == 0) return;
    const non_const_ptr = @constCast(bytes.ptr);
    @memset(non_const_ptr[0..bytes_len], undefined);
    self.rawFree(non_const_ptr[0..bytes_len], .fromByteUnits(Slice.alignment), @returnAddress());
}

All tests panicked because std.mem.Allocator.free⁴ writes undefined into freed memory before calling .rawFree. In Debug and ReleaseSafe modes Zig fills undefined bytes with a debug pattern (0xAA) to catch use-after-free bugs,⁵ which meant my test allocator saw non-zero bytes.

The solution? Bypass the layer altogether and call .rawFree() directly.⁶ Now, we can verify zeroization at the layer closest to deallocation.

mem.Allocator.free does what it does for a reason, and I don’t skip it lightly. The name .rawFree is meant to make you nervous. For example, in Debug or ReleaseSafe mode, setting all bytes to undefined means you get some use-after-free protection that we could be losing by calling .rawFree instead.

However, in this case, I felt comfortable with this choice after familiarizing myself.

The first three lines of Allocator.free are because memory can be anytype, so the code verifies memory is a slice, handles a potential sentinel, etc. This the only function I call .rawFree in is .deinit, which only accepts slices. No @typeInfo magic needed.

What about the @memset? Zecrecy uses standard library’s crypto.secureZero function to wipe memory, which uses techniques intended to avoid optimization removal. However, those techniques have their own flawed history and corner cases. Test reinforcement is always valuable, never take security for granted.

Outside of actually calling .rawFree, the only change I is to return early if the data length is 0. This has no security/correctness benefits but offers a small performance bump by avoiding the virtual function call when it’s not needed. Due to the nature of the library (how often will someone will be initializing a secret of length 0?), I think even if I had skipped this no-op opportunity, the .rawFree call is fine.

And boom! All tests pass:

~/zecrecy> zig test src/secret.zig
All 15 tests passed.

With each one, I can feel a tiny bit better knowing all secrets were actually zeroed before they were freed.

Conclusion

The effort required was low for the value gained, I’ve barely scratched the surface of using custom memory allocators to improve testing.⁷ Reducing the “black box” opaqueness that sometimes hangs like a fog over unit tests is a nice tool to have. I’m optimistic that Zig’s upcoming async/IO changes will provide similar opportunities to get surgical during testing.

And, of course, building test harnesses that enforce application invariants is far from unique to Zig. It’s a powerful technique with demonstrated value. Zig’s win, for me, is in making every decision explicit and shoving the reality of an implementation into your face at all times. You naturally think deeper about how to wield the verbosity in your favor.

Have any cool, weird, or otherwise nerdy uses you’ve gotten out of Zig’s memory management explicitness? Do you have a favorite method for testing your system’s security guarantees? Send me an email or a message!

TL;DR:

TIL: compile with a locally patched Zig standard library via `--zig-lib-dir`

Eli Grubb — Fri, 25 Jul 2025 00:00:00 GMT

alt-title: find a way to narrow down the scope of your comparisons (and compilations).

While benchmarking the impact of Zig removing inlines from their standard cryptography library, my naive initial approach compared the changes from the perspective of the Zig project as a whole, not just the standard library itself. In reality, this meant every time I modified a Zig crypto implementation while experimenting, even just to call std.debug.print, I was rebuilding the entire project from source.

The Zig build system feels powerful and I’m planning out a new project to coat myself in all the grimy details¹. In the meantime, I got started with Loris Cro’s excellent Zig Build System Basics video, and the existing documentation. I followed the Zig wiki’s instructions for building from source on both mac and linux.

But, of course, there’s a flag for that:

$ zig build --help

Usage: zig build [steps] [options]

...

Advanced Options:
  ...
  --zig-lib-dir [arg]          Override path to Zig lib directory
  ...

We can input the path of the lib directory containing an instantiation of the Zig standard library and zig build will compile without extra steps or linking. Now instead of feeling stuck at the bird’s eye view of the entire Zig source, it’s much easier to quickly compare and contrast the small nudges and curiosities.

Just call:

zig build --zig-lib-dir ../zig-master/lib

Those familiar with the ziglang codebase may note that the directory for the standard library isn’t lib/, it’s lib/std/. This means that --zig-lib-dir allows us specify a lot more than just the standard library for injection, we’re just scratching the surface here².

Benchmarking changes is much smoother now. I simply point my script to the appropriate git worktree directory for each version and away we go!

Odd tcsh issue on alpine linux [UPDATED]

Eli Grubb — Mon, 05 Feb 2018 00:00:00 GMT

I’m currently working with alpine linux, trying to port some existing systems over to this minimal OS. It’s not an easy task. Currently my issue is with tcsh. It seems that tcsh is triggering some kind of core dump every time it is called. Here’s the output when it’s called:

(nil) current memory allocation:                                                                                
free:       0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0                       
used:       0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0                       
        Total in use: 0, total free: 0                                                                          
        Allocated memory from 0x55bd1509d000 to 0xffffffffffffffff.  Real top at 0x55bd1509d000                 
nbytes=64: Out of memory

I’ve been googling and going through alpine’s bug tracker but there’s no mention of tcsh in there system beside the day they committed it to their repository. It is worth noting that the package is currently in the testing branch of their apk package manager so there were no guarantees. It is interesting that there seems to be literally no evidence anyone else has run into this issue. Also somewhat intriguing is the fact that the alpine maintainers have no documented interest in moving tcsh to a stable branch anytime soon. I agree that it’s probably an outdated tool, but still popular enough it should have higher priority.

If anyone has any potential solutions shoot me an email.

UPDATE (February 12, 2018):

Having submitted the bug to the Alpine Linux issue tracker, I received a pointer to a patch in the openwrt project that seems to fix this issue. At the very least, tcsh actually boots and seems to function as a shell once it’s applied. I’m going to work on either getting the patch committed to the actual alpine package repo or build a custom tcsh for my system with the patch.

UPDATE 2 (February 27, 2018):

This patch has now been committed to alpine’s apk repository. This took a little more effort than expected, and there’s definitely still some open issues with the package before it can be moved from the edge branch to the main package repo branch. As referenced in the linked github PR, the musl libc appears to break a lot of basic functionality in tcsh. As you can see here the number and nature of regression tests failed is somewhat alarming. However, by hand testing shows that basic functionality needed for my project works. Who knows, maybe it will convince some old-timers to switch to a more modern shell like bash ;).

It does however, bring up an interesting conversation about whether there is sufficient motivation for these issues to ever be fixed. The last few remaining active tcsh developers have no desire to work to get their package working with an upstart libc like musl and musl developers have no motivation to work on supporting a shell none of their target users actively rely on. I think my work has fallen into a rather small demographic.

Alpine Package Names

Eli Grubb — Wed, 08 Nov 2017 00:00:00 GMT

As part of my research work in the Flux Research Group I’ve been working very closely with Alpine Linux. My current project involves translating a bunch of scripts for building a particular Ubuntu image into the same process for an Alpine image. As such, I’ve had the fun of finding packages in Alpine’s apk package manager that are currently being installed through apt-get. This is just a quick post to serve as a reference for inconsistencies I’ve found across the two, and the packages I installed instead.

Ubuntu (16.04)	Alpine (3.6)
libtool-bin	libtool
libssl-dev	openssl-dev
libboost-dev	boost-dev
iputils-ping	iputils
perl-modules	???
libwww-perl	perl-libwww
tcsh	tsch@testing*
ksh	mksh
brctl	bridge-utils

* tcsh is currently only found in the testing branch of apk. You have to add the link to the testing branch to /etc/apk/repositories as described here.

If anyone has any corrections, questions or additions to this short list feel free to reach out to me on twitter or by email.

gitignore_global

Eli Grubb — Fri, 02 Oct 2015 00:00:00 GMT

I need git for every change I make anywhere on my computer apparently.

Today while working on a project for CS 3500 I ran into a curious bug that made me wonder what in the world I was thinking at the time.

In trying to commit some .dll files as resources to my Visual C# app in Visual Studio I came to the odd realization that they weren’t actually included in my repo. Curious.

It got even weirder when I manually right-clicked and added it to my git. Yet when I clicked commit I got an error claiming that I wasn’t committing any files even though they were both RIGHT THERE. Curiouser and curiouser.

Finally I was able to debug my issues when I happened upon some extremely helpful git commands thanks to this stack overflow post.

Using the described techniques I found a file in my Documents folder (no where near or associated with my C# development) titled “gitignore_global”. This document explicitly set git to ignore several visual studio specific files including, you guessed it, *.dll.

I renamed the file and what do you know? It all committed nice and easy now! WOO HOO!

I still have no idea why I created that file in the first place. And why in my Documents folder that has nothing to do with any of my development projects??

I’m still scratching my head. Oh well. I guess that’s better then banging it against my desk in frustration.

Sometimes even the best technologies are just plain dumb.

Testing Class Libraries with C# in Visual Studio

Eli Grubb — Tue, 08 Sep 2015 00:00:00 GMT

When building a Class Library for my CS 3500 class we are required to build our own little testing suites on our machines. These tests aren’t to be uploaded to the GitHub repository for our project, but obviously it is necessary if we want to build a functioning program. As this is my first foray into the world of C# and, more importantly, Visual Studio, I found it slightly difficult to figure out out to write even rudementary Console Application tests for my Class Library. This post is to act as a reminder as to how to successfully add a ConsoleApplication Project to a solution and use it to test a Class Library.

Note: This was done using Visual Studio 2015 Enterprise and any differences in your setup is probably a result of this

Once you have your initial Class Library created with a class and method you would like to test, you will find yourself to run any tests within the same project. Due to Class Librarys compiling down to .dll files and not something like a .exe there is no way to run that project directly. Instead you need to create another project that does compile down to a .exe and have it reference your Class Library

The first thing you will do is select File -> New -> Project…

In this case I am testing my project using a Console Application so I will choose Visual C# -> Windows -> Console Application as my application type.

It doesn’t matter what your project’s name is, but for simplicity I suggest using the format Test. In this case since I’m working on a Class Library called Formula Evaluator I’m naming my Console Application FormulaEvaluatorTest.

Make sure it’s in the same location as your Class Library Project and add to the existing solution to ensure continuity.

After you create this new project it will open up it’s Program.cs file which contains what this whole process is about, the Main Function. Now you have a program that can actually execute operations. However, you are not quite done.

To ensure you have access to your Class Library in your new project, in the Solution Explorer right-click on your Console Application and select Add -> Reference…

Select Projects -> Solution from the left hand column and find your Class Library in the center pane. Make sure that it’s checkbox is checked, then hit OK. You should now be able to write the line using MyClassLibrary in your console application without any errors.

You are just about done. But wait! When you attempt to run your Console Application, you probably end up with this error:

Your solution is automatically choosing your Class Library to run when you run the solution as a whole and this just doesn’t work. Like we discussed earlier, since Class Libraries compile down to a .dll file you cannot run it. You need to run the .exe file that your Console Application compiles down to.

To change this you simple right-click on your Console Application and select Set as StartUp Project

Now when you hit the Start button on the top toolbar to run your solution, it is your Console Application that will run.

Happy (C#) Hacking!

UPDATE 9/16:

I finally learned how to do this with unit tests. And I feel rather silly now that I’ve been taught. Simply replicate my process but instead of selecting a console application, select the “Tests” category on the left and choose a Unit Test Application. Unit Tests are so much better than testing with a Console Application and I’m not looking back. But that’s mainly because of how silly I feel not knowing how to use unit tests before.