William Floyd (high-quality images)

Notes on LLMs - Part 2

William Floyd — Fri, 06 Nov 2026 00:00:00 +0000

Some notes on how I’ve found the most use out of LLMs. The specifics of the tools honestly change so fast, I won’t get into the details too much, but here’s a look at some themes I’ve found. Note that some of these principles are likely helpful even without LLMs, but are especially important with them.

TL;DR

Know your actual goal, and when “good enough” is all it needs.
Get something working first, and make the feedback loop as fast as you can so each iteration runs on its own.
Turn anything you repeat into a script: scripts are inspectable, reusable, and outlast cheap tokens.
Do homework on prior art and primary sources before running an LLM on a large or opaque task.
Reshape a task into a common, well-trodden form, so the novel work is small and contained.
Get both ends working, then use the LLM to fill in the step between them.
With profiling and low stakes, LLMs are good for trying out speedups.
LLMs are a hammer; not everything is a nail.
Pin down how you’ll measure features, performance and correctness, early.

Keep the “Main Thing” the Main Thing

Know what you’re looking to achieve when you set out on a project.

This depends heavily on the criticality of what’s being done, but know when “it ain’t stupid if it works” applies. For example, with my work in Part 1 getting a phone and then a car’s head unit running Linux, the goal was just that - getting Linux running.

Non-goals included:

sharing every random script used
documenting every step
doing it “right”
mainlining modifications

This sounds terrible, perhaps, but bear with me. The goal is to get mainline-ish Linux running on these devices. No-one else has done this, to my knowledge - not because it’s impossible or even hard for the right person (it is hard for me), but because I happen to be the first attempting it.

I assure you I had rather be the first man here than the second man in Rome.

— Julius Caesar (Wikiquote)

Put another way, if you can’t be the best, at least be first.

This is not, however, meant to disparage taking the time to do good work. On the contrary I value good work very much, and hope to do as much of it as I can. But sometimes what I need is “good enough” work to achieve a goal.

LLMs are a tool used to achieve this goal for me. Were my goal to simply write lines of code, there are myriad ways of fulfilling that goal, but rest assured I would not achieve the goal of booting mainline-ish Linux as quickly as I did with LLMs.

Make It Work, Make It Right, Make It Fast

Make it work, make it right, make it fast.

— attributed to Kent Beck (c2 wiki)

LLMs are fairly good at the first (and sometimes the third, see more here) part of this phrase. Run an LLM in a loop against a clear goal, and you can often brute-force something working, however hacky it might be.

Again, this may sound terrible, but I find it much easier to have a working end-point to work backwards from, than to do everything “right” from scratch. This is especially true of things like embedded Linux bringup, where simply getting a booting system with remote access is so much more helpful than any amount of “doing it right” blindly could ever yield. Once you get it working, it’s much easier to get it working right.

When using LLMs it is especially important to close the feedback loop as quickly as possible, so the tool can iterate automatically. For example, in Part 1, I ensured there was a script that automated as much of the USB-to-boot cycle as possible. See also xkcd 1205, “Is It Worth the Time?”. Which leads me to my next point…

Produce Durable Goods

Make scripts for everything you do more than twice. LLMs have a tendency to produce inscrutable one-liners for routine tasks, so I find it helpful to step in to ensure durable scripts are made instead, especially if I know I’m going to do the task often. These scripts don’t need to be especially good, but the fact they’re at least written down in one place and inspectable is critical. They can be committed, reviewed, and tweaked as time goes on. These scripts, once sufficiently worked through, serve to reduce the cognitive load on humans and context usage of LLMs, which then simply run a small command to do a rote task instead of reinventing the wheel.

These cobbled together scripts may be a rat’s nest as a whole, but if they work, they inch you closer to your goal. Once that goal is achieved, it is at least repeatable. And once it’s repeatable, it can be made more “right” (see above).

These “durable goods” will outlive the cheap LLM tokens. Once the true cost of operating LLMs at a loss funded by gobs of VC money is brought to bear, we will still have these outputs. Ultimately, the less you depend on LLMs being cheap, the better. And if they do remain cheap (or get cheaper), then all the better I guess…

Prior Art and Primary Sources

It is useful to survey prior art and primary sources before just setting LLMs running on a given task. For example, in Part 1, tracking down the somewhat obscure Mobis open source portal got me a vendor kernel tree. Perhaps an LLM would have found it eventually (and I’d still have had to download it myself), but by doing at least a modicum of up-front homework, a lot of time can be saved. In this instance, the LLMs could then derive vendor changes for the SoC in question and port them to a new mainline branch, something that would take me far, far longer.

Intermediate Formats and Task Normalization

Massage your tasks into something that is like something that is already commonly done. Once a common intermediate format is achieved, you will have a much easier time of making progress. This is especially true when reverse engineering, but may still prove useful in other projects. For example, converting an obtuse API spec from a 500-page PDF into an OpenAPI spec that is 95% complete with the last 5% captured in prose descriptions yielded better results with LLMs than just using the PDF directly.

I might call this “task normalization”: breaking a project down into the most idiomatic steps such that LLMs need to do the least novel work, or at least that the novel work is well bounded and concentrated into one domain.

Fill in the Blank

A theme alluded to above is the idea of filling in the blank - you get the bare minimum working, and flesh out the in-between later. This is not always appropriate, but when it is, it allows LLMs to serve as a powerful tool to get some momentum on a goal. The “blank” need not be a full project, it may be some increment - so as usual, it does well to at least think through what is required to solve a particular problem.

Take, for example, camera support for the Moto G Play in Part 1. I had libcamera, stock camera calibration files, and a working sensor. The “blank” I had to fill was converting these stock calibration files into something that libcamera could at least begin to use for better camera support. The goal (in keeping the “main thing” the main thing) was a working camera that at least looked halfway decent. The useful increment to get there was to make a tool to read stock calibration files and convert them into something that libcamera could read, and make libcamera modifications to support them. That way I could go from stock calibration to a working camera without redistributing stock files.

This camera case is a good low-stakes example - getting it wrong is both immediately evident and harmless: it’s simply a “bad” picture, no harm done. Thus, doing it “right” isn’t terribly important - if it looks good, it is by nature an acceptable result. It also lent itself well to an iterative approach, wherein I used LLMs to trawl GitHub for relevant source code (i.e., the vendor-distributed camera stack from Qualcomm), read calibration files, approximate that same feature (if new) in libcamera and then test on a live device. The tight development loop meant I could use LLMs to work against both captured RAW images and live camera feed and start to zero in on an acceptable result. For more subjective/temporal failures that required human review (unstable brightness, focus breathing) it was still easy to use LLMs to try things, with occasional nudges in the right direction.

I Am Speed

Once you’ve done the “make it work”, you should do the “make it right”, but if it’s low stakes and you’ve instead done the “make it right (enough)”, then why not go ahead and do the “make it fast”? LLMs, when combined with good old fashioned profiling, are great for testing things out.

The key here is to scope the problem: find a user experience that is slow, make sure it can be profiled (in itself something that LLMs could help set up), then you can dig in. It’s a low stakes way to try things, and see some of what’s possible for a given problem. For example, take this PR to flacoutcpp, a tool that searches for the smallest FLAC encoding of a file. Profiling put 93% of its exhaustive mode’s runtime in two loops, and reworking just those made it about 3x faster with byte-identical output. It also added a search level about 90x faster than exhaustive mode, for about a quarter of a percent in file size. Lossless audio kept the stakes low: every change could be checked by decoding and comparing against the source.

Hammer Meets Nail

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

— Abraham Maslow, The Psychology of Science (1966) (Wikipedia)

LLMs are a hammer; learn to tell what is actually a nail. This, depending on experience and personality, will likely be difficult (it is for me). As LLMs get better, it will become increasingly difficult.

I have no solution for this, and I am not a good example of doing this anywhere close to right. For now, I’m seeing a lot of nails.

Treasured Measurements

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.

— Lord Kelvin, “Electrical Units of Measurement” (1883) (Wikiquote)

Put another way:

You can’t treasure what you can’t measure.

You must be able to measure several things for a successful project, some of which include:

Features
Performance
Correctness

Nail these down as soon as you can, and you’ll have more success. Acceptance criteria exist for a reason. At the same time, if you learn something along the way and the goalposts must shift, so be it - but be discerning when doing so; LLMs often make it feel like I have more bandwidth than I really do.

Still a mixed bag, as in Part 1, but here we are anyway.

Part 3 someday, likely once I get a bench setup for a spare EV6 head unit (working on getting 64bit boot).

Notes on LLMs - Part 1

William Floyd — Sat, 17 Oct 2026 00:00:00 +0000

I had written a draft of an anti-LLM post some months ago, and ended up not posting it. Since then, I’ve used these tools much more during my day job, as well as hobby projects. I’ve had many thoughts about LLMs, and their impact on my mental health and ability to learn (or the stunting of said growth). I’ve also experienced many cases where it is undoubtedly an amazing tool that has enabled me to learn new things by reducing the barrier to entry for new fields.

In all, it’s been a mixed bag, and I wish LLMs never existed.

But the technology is here now, so I thought I’d show some of the things I have managed to do aided by these tools. In line with not letting perfect get in the way of good/done, I’ll probably post updates in Part 2 and so on.

Linux Hacking

The Linux kernel is huge, complicated, amazing, and not something I have experience in modifying. I am however quite comfortable compiling software and figuring out how the pieces fit together (eg here and here). That being the case, I’ve gone ahead and tried a couple projects with kernel related hacking.

Samsung Galaxy S9 - `starqltechn`

My old phone collecting dust, so why not try to get it running Nura?¹ There’s already a working port, but lots of peripherals don’t yet work, so I figured I’d start there.

Long story short, using LLMs I was able to get:

Working camera+flash
Aperture control
Manual focus
WiFi+BT (tracked down an existing patch)
DisplayPort output

My Code

Kernel tree
aports (spread across branches)

Vendor Resources

Level 3 service manual (SM-G9600)
Samsung 4.9 kernel source (Droidian’s tree)
Stock firmware dump

Motorola Moto G Play (2024) - `fogona`

Fresh on the success of getting progress on an older phone, I decided to find the cheapest phone that a US resident can just go out and potentially install Linux on. At this time, it’s the Motorola Moto G Play (2024), which can be had for $100 USD new in box, carrier unlocked and bootloader unlockable. There’s already SoC support, but it’s somewhat early.

After much back and forth, I was able to get:

GPU support
Video decoder acceleration
Cameras with PDAF and stock tunings
WiFi, BT
Modem (data, calls, texts/mms)
GPS, accelerometer, compass, rotation, lux, proximity

My Code

Vendor Resources

Level 3 service manual (XT2413)
Kernel: Older, more complete, Latest
Device tree: Older, Latest
Subsystem device trees
- display: Older, Latest
- camera: Older
- audio: Older, Latest
- nfc: Older, Latest
- bt: Older, Latest
- graphics: Older
- video: Older
- wlan: Older, Latest
Drivers
- Motorola modules: Older
- Camera: Older
- Graphics (KGSL): Older, Latest
- Video: Older

PRs that resulted:

Projects that resulted:

bootmanager

Such an inventive name, I know… A Slint GUI + Go backend tailored for Linuxboot+u-root setups, meant to be a beautiful alternative to uboot. Supports touchscreen, SSH, fastboot, LUKS decryption, usage in place of unl0kr, and a web GUI built in WASM from same Slint.

▶ Watch the video

godot-maps

Comaps doesn’t look great on Linux, and I couldn’t find anything else that did what I wanted, so I made a quick and dirty navigation app built on Comaps backend using Godot. It compiles to many platforms too, including MacOS and Android.

go-msm-firmware-loader

Go port of msm-firmware-loader, used by bootmanager

go-qbootctl

A static Go replacement for the A/B slot subset of qbootctl so it works in an initramfs, used by bootmanager

go-chromatix2libcamera

Translates Qualcomm Chromatix tuning files to something libcamera (or a fork thereof) can interpret.

go-unbrick

Swiss army knife that I use to inspect stock firmware images, any files contained therein, and to make some files to help with bricked devices.

go-qshrink4

A reader for Qualcomm QShrink4 hash databases (.qdb / .qsr4), mapping the hashes in QShrink’d modem diag messages back to their format string, file and line.

2024 Kia EV6 Head Unit

New (to me) car, new project! Once I got the stock system image (download from Kia, community knows how to decrypt), a root shell on the running unit (others already figured this out), and the vendor source, I was off to the races.

Given the ancient (3.18.24, from 2015) vendor kernel (for Android Kitkat!), I used LLMs to do minimal mainline porting for the quad Cortex-A53 Telechips TCC803x SoC, which turned out to almost work first try. With a bit of back and forth (and much USB swapping), display out was achieved and I could feed log messages back into the LLM and iterate on fixes. Note that stock was never reflashed, this is a kexec cycle that uses USB data only. The old Android version is in some ways a blessing in disguise, because older systems are in some ways much easier to bend to your will.

I figured the quickest path to something useful was to get bootmanager running, which turned out to be trivial.

Still, after ~3.5min, it would reboot. So using LLMs I logged serial transfers from Android and did some decompilation of the MICOM firmware (seems to be some kind of PowerPC VLE chip) and Android apps to see what signals I needed to respond to, and found a hung SoC check that I can respond to to keep from being reset. At the same time I found signals for things like power draw of climate, battery state, etc. (I still need to map this out properly).

Working:

Display
I2C
USB
eMMC
WiFi (and soon, BT)
Touch
Serial
MICOM keepalive + Vehicle data
re-kexecing back to stock Android

Does it run DOOM?

If you saw the cover image, you’ll know. Of course it does!

▶ Watch the video

Vendor Resources

Mobis open source: “AVNT STD G50 WIDE R6” for “EV6 (CV1)” (AVNT_STD_G50_WIDE_R6.zip, 565 MB)

My Code

Kernel tree

Formerly PostmarketOS ↩︎

DIY Solar Battery Backpack for the XTEINK X3

William Floyd — Wed, 30 Sep 2026 00:00:00 +0000

I love my XTEINK X3 (running Crosspoint), and wondered if I could make a quick and dirty solar backpack for it. I planned a better writeup but here I am anyway not letting perfect get in the way of done.

Pictures

Battery

Mold/Pour

Pour Results

Magnets

Final Result

Notes

Thin batteries are hard to get in low quantities
A thinner solar panel would be really helpful
I need to get my 3D printer running again - a printed shell + fixture to hold pogo connector would save a lot of work
Next time will avoid the epoxy itself being the rear surface - sanding flat was a lot of work, and imperfect
Charge time and efficiency are likely quite poor
Silicone tape seemed to work nicely for the battery, but robustness/actual affect untested
Getting the reed switch positioning correct is a pain when the backpack itself needs to carry it’s own magnets - maybe look at pogo-pin style switch?
A custom PCB could help reduce thickness/complexity
- As a fallback, a carrier PCB for modules would be trivial and inexpensive
I started designing a CNC case for this with the intent to do another spin, but a few things put it on the backburner:
- Space constraints are tight, case will likely still be thicker than I want
- The X3 already has excellent battery life, making this a novelty more than a useful item
- I have too many other projects in-flight

BOM

Part	Role	Notes
5V 150mA polycrystalline solar panel (~89 × 61mm)	Charging input	Close to the footprint of the X3’s back - I very much doubt 150mA claim
CN3163 mini solar charger board	Battery management	Linear solar-tracking charger, 4.4-6V in, 4.2V out
TPS63020 buck-boost module	5V output	Fixed 5V via a solder jumper, has an EN pin
303450 LiPo pouch cell, 3.7V 500mAh	Energy storage	Roughly doubles the X3’s onboard capacity
Molded reed switch (N/O)	Magnetic power switch	Enables the boost converter only when attached
4-pin 2.54mm magnetic pogo connector	Output to the X3	Matches the X3’s charging pin pitch
BSI slow-cure epoxy	Potting
Silicone tape	Battery sock	See below

I used silicone tape to seal the battery so as to prevent epoxy doing anything to the battery - I’ve not tried encasing a cell in epoxy directly, but the heat and shrink/expansions scares me and this is what I landed on that would allow some amount of squish.

Antlion ModMic Wireless Battery Swap

William Floyd — Fri, 24 Jul 2026 00:00:00 +0000

I’ve used an Antlion ModMic Wireless since I started my first job (where I regularly took calls 6+ hours a day). Over time, its runtime has dropped from a full day to about half, and Antlion doesn’t sell replacement batteries, so officially my only fix was a new mic. But it’s held together with just three screws, so swapping the battery is easy enough (took me 15 minutes).

The base of the mic is a two-part plastic body held together by three small screws, with nothing glued.

The cell is a 502030 LiPo cell (3.7V, 250mAh, dated 2022.04.22), stuck with double-sided tape and soldered to two pads. That “502030” is the cell’s size (5 x 20 x 30mm), which is all you need to find a replacement. I ordered a replacement 502030 from Amazon ($8 after tax). Unfortunately there’s no space to choose a larger size.

Mine came with a JST plug, so I snipped it off (don’t cut the wires at the same time!). The old cell comes off by desoldering its two leads and working a piece of cardstock underneath to release the double-sided tape without damaging the old cell. The replacement goes in the same way - leads soldered to the same pads, and taped down in the same spot. Screw it together the same way it came apart.

Nice Things

William Floyd — Fri, 26 Jun 2026 00:00:00 +0000

I grew up in a decidedly under developed country¹, and with that came under developed internet infrastructure - for the longest time it was all but non-existent. Then cellular internet slowly rolled out, but was (or is still) slow, expensive, and unreliable. I learned to program in this environment, where heavy dependencies had a very tangible cost associated with them - both time and money.

I would even build elaborate Excel spreadsheets just to be able to track my remaining data allowance before my plan would run out, and meter myself to make it stretch the whole distance. Ad-blocking was a must - good luck loading anything heavy on patchy 3G speeds. Video streaming was squarely out of the question. At times, I would block all images entirely just to be able to browse text tutorials more easily.

And yet, I was productive. I learned to be patient, to think rather than be lazy and web search. I would read local docs, I would plan ahead, work in parallel with downloads, and budget my resources for maximum benefit.

I have since moved to a more developed country², and enjoy the benefits of a well developed internet infrastructure³. But old habits die hard, and I’ll not allow myself to become lazy when developing my own site/software and bloat it unnecessarily. It might not matter so much to me, but I’ve been there long enough to care about optimizing content delivery to be as lean as possible.

And so it is that I want to document some of the approaches I have taken to minimizing site loading time and data usage.

Tradeoffs

You can’t have everything, so I have a few preferences I make decisions around:

If new browsers universally support a useful feature, I’ll use it, even if slightly old browsers don’t support it
Where something can be precomputed, I prefer to do so - clients shouldn’t be responsible for any data processing if possible
RSS reading must be a first class citizen

(Images) Under Pressure

Lossy

JPEG is great, and universally supported, but somewhat inefficient. Yes, you can optimize (jpegoptim, etc), but really there are better codecs. WEBP is decent, and worth a try, but really, we should go all the way and implement AVIF. It’s well supported⁴ these days, so it’s the best option for efficiently serving images.⁵

Bracketing

So now the question is, how to serve the most optimal image for every given client? Part of my solution is to pre-encode many size brackets of each AVIF image for use in srcset⁶, so the client will pull the closest to 1:1 scale image possible. This gets us pretty far along towards an optimal page - images look crisp, and so long as we chose a reasonable encoding quality there is limited visual degradation.

For those that want it/can afford it, I also encode -hq versions of each image with much better quality at the cost of increased data transfer.

Visual Quality vs Encode Quality

“So long as we chose a reasonable encoding quality” is doing some heavy lifting. In short, constant encoder quality number != constant visual quality. Picking one number for the whole site leaves some images bloated and others very much degraded. To this end, I measure SSIMULACRA2 per encode and tweak encode quality until I reach my target visual quality, as measured against the lossless resize of the source (resize quality != encode quality). The site ships two visual quality targets, 65 for the bandwidth-tuned ladder and 85 for the -hq siblings.

(Soul) Searching

Initially, I had a naive approach, probing quality upwards until the score cleared the bar. That worked but was slow - at -s 0 a single large encode is tens of seconds, and every probe is an encode plus a decode plus a metric run. Binary search helped, and for a 30–85 window cost about six probes. But bisection throws away useful info about each probe, so the search is a damped secant root-find on score(q) - target.

Measured on this corpus:

Content	Near target	SSIMULACRA2 points per quality point
Photograph, 4:2:0	65	1.0 – 1.5
Photograph, 4:2:0	85	0.33 – 0.45
Screenshot, 4:4:4	anywhere	0.19 – 0.29

The table above only ever seeds the first step. From the second probe onward the slope is the secant through this image’s own two measurements, so the search learns the specific curve it is standing on rather than trusting an average. Two guards: the search brackets highest quality that failed, and lowest quality that passed, clamps every proposal strictly inside it. So if a proposal lands on a quality already probed, it bisects the remaining bracket instead of wasting the encode. It’s done when a winner passes and the quality one step below it fails, proving nothing is left on the table.

To help future encode runs, every encode appends a row to a log, and a least-squares fit over that log gives a starting-quality for any given encode. I refit after every encode because it’s far cheap enough than any probe encode. The resulting four numbers live in src/q-model.env, which is committed, so a fresh clone inherits a converged search instead of relearning it from scratch.

By way of Example

Normal vs `-hq`

That patch is cropped out of the source photograph before anything encodes it, and then handed to the ordinary build twice — once at 65, once at 85. What you see below is those two files, served exactly as the encoder produced them: normal (SSIMULACRA2 65) on the left, -hq (85) on the right.

3 KB against 6 KB for the same 240×240 patch. Both ship at native scale, one file pixel per encoded pixel, and your browser enlarges them with image-rendering: pixelated

As seen on this site

The image below ships both encodes straight from the usual responsive build. Use the image-quality toggle in the corner, which cycles LOW → MED → HI, to switch between them: MED is the bandwidth-tuned default, and HI swaps in the high-quality set. LOW is just the smallest sized wrung of MED.

At full resolution the normal encode is just 489 KB, against 2.2 MB for the high-quality one. I personally can’t see much of a difference in normal viewing conditions. At the smallest wrung (400px wide): the normal encode is just 4 KB, against 12 KB for the high-quality one. Note that clicking the image loads the largest resolution version of the image in whatever quality setting you’re in, so you can zoom in there if really want.

What this page actually cost you

Everything below is measured in your browser, for this visit. It updates live, so flip the quality toggle to and watch the image totals change the as browser refetches (likely from cache) the alternative quality.

Not shown in feeds: a live table of what this page cost your browser. View it on the site.

Lossless

PNG is great too, and likewise universally supported, but also inefficient. Again, yes, you can optimize (oxipng, etc), but also again, better codec. This time, it is WEBP. Similarly, it’s well supported.⁷

Not everything is a photo

To help flavor the choice for WEBP for lossless images, here’s an example:

Every column is its format’s best effort on identical pixels: one canonical PNG is rendered first and handed to each encoder, cwebp -z 9 and avifenc -s 0 are both at maximum, and the PNG column is itself run through oxipng with Zopfli (which is dog slow, even worse than any other optimizer we see here).

Content	PNG	WebP (lossless)	AVIF (lossless)
Photograph	1.4 MB	1.2 MB	1.3 MB
Screenshot	274 KB	104 KB	436 KB
Plot / graph	29 KB	16 KB	75 KB
UI capture	24 KB	14 KB	63 KB

WEBP’s lossless mode wins every category, so the build routes photographs to lossy AVIF and everything else to lossless WebP. WebP’s lossless mode, VP8L, shares almost nothing with its lossy path, while AVIF has no equivalent. AVIF’s own working group says so, having weighed a reversible-transform fix worth 7–15% and declined it as not worth a spec extension.⁸

Quick maffs

A handful of posts here have mathematical notation, and the usual way to render it in a browser is KaTeX. On a fast connection you might never notice, but it’s not acceptable for my goals. At first, I vendored KaTeX to avoid overhead, which did help, but there’s a better option: MathML.⁹ Hugo’s transform.ToMath can emit presentation MathML instead of KaTeX’s HTML, eg:

y = m x + b

So 21.7 KB of CSS and roughly 275 KB of WOFF2 are gone. What ships now is the markup itself and a single inline CSS rule, emitted only on pages that actually contain math, giving block equations some breathing room and letting wide ones scroll instead of overflowing on a phone. One wrinkle, transform.ToMath includes an <annotation> holding the original LaTeX so every formula showed up twice in RSS. The feed template strips the annotation, the page keeps it.

Twisting my own words

Images will always be the heavy part of a post, but we still have some juice left in the text. My reverse proxy (Caddy) used to compress all of it on the fly, with zstd or gzip at their default levels, which was fine, but we can do better - namely, compress the snot out of text files at build time.

The docker build now compresses every text file over 1 KB three ways, each at maximum effort:

brotli -Z ¹⁰
zstd --ultra -22 ¹¹
zopfli --i4

For a representative .html file, we get the following tradeoff on zopfli iteration count:

And across the entire site, we find:

We got most of the value of zopfli with only a few iterations, and this is only used when clients don’t take zstd or brotli, which are better anyway.

The container that serves this site now uses Caddy, which picks the pre-compressed file matching what the browser accepts and sends it as is. My reverse proxy out front will just pass these compressed files straight through.

What it bought

Measured against the old setup, for a browser that accepts brotli:

Transferred	Before	Now	Saved
Home page (HTML + CSS + JS)	13.8 KB	11.2 KB	19%
Post list	13.5 KB	10.9 KB	19%
This post, including the charts' uPlot	48.8 KB	40.6 KB	17%
Every text file on the site	1,253 KB	959 KB	23%

Net result being roughly a fifth off, with no extra work for the server at request time or the client browser at all. The “before” column assumes Caddy’s default zstd level, roughly zstd -3.

Papua New Guinea ↩︎
USA, mostly ↩︎
At developed country prices, a whole topic in itself ↩︎
https://caniuse.com/avif ↩︎
JPEG-XL is neat but has limited browser support. ↩︎
https://caniuse.com/srcset ↩︎
https://caniuse.com/webp ↩︎
Still open as av1-avif#111, where the format’s own contributors concede that “the AV1 lossless compression mode has not been as finely tuned as the lossy modes” and that LZ77-style coding suits this content better. av1-avif#89 says “deciding we don’t care about lossless”, “since a purpose-designed codec for lossless like WebP-LL […] can usually outperform a hybrid block based codec like AV1”. ↩︎
https://caniuse.com/mathml ↩︎
https://caniuse.com/brotli ↩︎
https://caniuse.com/zstd ↩︎

Extracting Energy Data from Home Assistant

William Floyd — Thu, 09 Apr 2026 00:00:00 +0000

Couldn’t find a complete guide for this.

Install https://github.com/klausj1/homeassistant-statistics

Run like so:

Using this as the timestamp string: %Y-%m-%dT%H:%M:00Z

Once downloaded, to find what id you should filter to (you might already know the entity name from Home Assistant):

cat export_counters.json |
    jq 'map(.id | select(contains("consumption")))'

In my case, the file is large, so it’s best to pre-process the file and filter it to just the target entities values:

cat export_counters.json |
    jq 'map(select(.id=="opower:xxx_energy_consumption")) | first | .values' > export_filtered_counters.json

Now you have your power data, ready to process!

This is where things can get as crazy as you like, depending on your affinity for JQ.

Simple Route (CSV / Excel)

If you want to do stuff in Excel (or any similar product) you might want to further process this data:

cat export_filtered_counters.json |
    jq 'map(.datetime |= (fromdate | strftime("%Y-%m-%d %H:%M:00")))' |
    yq -o csv > export_filtered_counters.csv

This gets you a CSV with date strings that Excel can work with:

datetime,sum,state,delta
2025-04-05 00:00:00,3010.138800000007,1.041,0.0
2025-04-05 01:00:00,3010.948800000007,0.81,0.8099999999999454
2025-04-05 02:00:00,3011.637600000007,0.6888,0.688799999999901
2025-04-05 03:00:00,3012.369600000007,0.732,0.7319999999999709
2025-04-05 04:00:00,3013.131600000007,0.762,0.762000000000171
2025-04-05 05:00:00,3013.871400000007,0.7398,0.7397999999998319

You’ll probably just want the state and datetime column.

Draw the Rest of the Owl

I get carried away, and built out JQ functions to model my utility rate options. Ultimate flexibility, much easier than wrangling in Excel.

Script. Usage is like so:

cat export_filtered_counters.json | jq -f calculate.jq

(
    [
        153.90,
        118.72,
        84.34,
        80.71,
        101.59,
        116.60,
        136.78,
        192.62,
        207.16,
        144.40,
        135.87,
        114.62
    ]
    | add
) as $actual |
def isHoliday($dateTime):
    ($dateTime | strftime("%m") | tonumber) as $month |
    ($dateTime | strftime("%d") | tonumber) as $day |
    ($dateTime | strftime("%w") | tonumber) as $weekday |
    # New Year's Day: January 1
    ($month == 1 and $day == 1) or
    # Memorial Day: last Monday in May
    ($month == 5 and $weekday == 1 and $day >= 25) or
    # Independence Day: July 4
    ($month == 7 and $day == 4) or
    # Labor Day: first Monday in September
    ($month == 9 and $weekday == 1 and $day <= 7) or
    # Thanksgiving Day: fourth Thursday in November
    ($month == 11 and $weekday == 4 and $day >= 22 and $day <= 28) or
    # Christmas Day: December 25
    ($month == 12 and $day == 25)
    ;
# Examples
# RESIDENTIAL GENERAL USE AND SPACE HEAT - ONE METER: 2RS6A
def _residential_general_use_and_space_heat_one_meter_2rs6a:
    . as $dot |
    (
        group_by(.month) |
        map(
            (first | .month) as $groupMonth |
            {
                month: $groupMonth,
                state: (map(.state) | add)
            }
        )
    ) as $monthstates |
    $dot | map(
        . as $entry |
        ($monthstates | map(select(.month==$entry.month)) | first | .state) as $thisMonthstate |
        $entry |
        (.month >=6 and .month <= 9) as $isSummer |
        if $isSummer then
            0.10021
        else # Winter
            # That is, the rate is 0.06966 for the first 1000, and 0.06123 for everything past 1000
            if ($thisMonthstate>1000) then
                # Calculate the average charge given the 1000 split
                ((($thisMonthstate-1000) * 0.06123) + (1000 * 0.06966)) / $thisMonthstate
            else
                0.06966
            end
        end |
        $entry + {rate: .}
    )
    ;
# RESIDENTIAL GENERAL USE – ONE METER: 2RS1A
def _residential_general_use_one_meter_2rs1a:
    map(
        . as $entry |
        (.month >=6 and .month <= 9) as $isSummer |
        if $isSummer then
            0.10021
        else # Winter
            0.07735
        end |
        $entry + {rate: .}
    )
    ;
# 2RTOU, With Net Metering 2RTOUN
def _2rtou_with_net_metering_2rtoun:
    map(
        . as $entry |
        (.month >=6 and .month <= 9) as $isSummer |
        (
            (.weekday > 0 and .weekday < 6) # Weekday
            and (.hour >= 16 and .hour < 20) # 4pm-8pm
            and (isHoliday(.datetime) | not) # Not holidays
        ) as $isOnPeak |
        (.hour >= 0 and .hour < 6) as $isSuperOffPeak |
        if $isOnPeak then
            if $isSummer then
                0.26838
            else
                0.20151
            end
        elif $isSuperOffPeak then
            if $isSummer then
                0.03834
            else
                0.02879
            end
        else # Off-Peak
            if $isSummer then
                0.07668
            else
                0.05758
            end
        end |
        $entry + {rate: .}
    )
    ;
def _rtou2_with_net_metering_rtou2n:
    map(
        . as $entry |
        (.month >=6 and .month <= 9) as $isSummer |
        (
            (.weekday > 0 and .weekday < 6) # Weekday
            and (.hour >= 16 and .hour < 20) # 4pm-8pm
            and (isHoliday(.datetime) | not) # Not holidays
        ) as $isOnPeak |
        (.hour >= 0 and .hour < 6) as $isSuperOffPeak |
        if $isSummer then
            if $isOnPeak then
                0.2725
            else # Off-Peak
                0.0681
            end
        else # Winter
            if $isSuperOffPeak then
                0.0397
            else # Off-Peak
                0.0793
            end
        end |
        $entry + {rate: .}
    )
    ;
def _apply_rate:
    map(
        . + { cost: (.state*.rate)}
    );
# Monthly Costs
def _add_monthly_cost:
    . + 14.25*12;
def _add_rate_fees:
    .
    # | map(.rate *= 1.287) # TAX ADJUSTMENT
    | map(.rate += 0.00103) # PROPERTY TAX SURCHARGE
    | map(.rate += 0.01084) # TRANSMISSION DELIVERY CHARGE
    ;
def _cost_adder:
    map(.cost) | add;
def _analyze_by($grouperField; analyzer):
    group_by(.[$grouperField]) |
    map(
        (first | .[$grouperField]) as $group |
        {
            $grouperField: $group,
            data: analyzer,
            # total_cost: _cost_adder
        }
    );
def _flatten_analyze($obj):
    map(
        . as $entry
        | if (($entry.data | type) == "array") then
            $entry.data | _flatten_analyze($entry | del(.data))
        else
            $entry + $obj
        end
    ) | flatten;
def _flatten_analyze:
    _flatten_analyze({});
def _is_outlier($state):
    # Heuristic
    $state > 6 or $state < 0;
def _fix_outliers:
    (
        map(select(_is_outlier(.state) | not))
    ) as $withoutOutliers |
    (
        $withoutOutliers
        | group_by(.hour)
        | map({ (first | .hour | tostring): (map(.state) | add / length) })
        | add
    ) as $hourAvgs |
    map(
        if _is_outlier(.state) then
            . + { state: $hourAvgs[.hour | tostring] }
        else
            .
        end
    )
    ;
(0.550) as $load | # kWh
(0.920) as $RTE | # Round Trip Efficiency
(0.850) as $SoCWindow | # State of Charge window, eg 10% to 90% = %80
2.875 as $hours | # Intended duration of battery runtime
($hours*($load/$RTE)/$SoCWindow) as $batterySize |
def _adjust_usage:
    map(
        if (.hour >= 16 and .hour < (16+$hours)) then
            .state -= $load
        elif (.hour >= 0 and .hour < ($hours)) then
            .state += ($load/$RTE)
        else
            .
        end
    )
    ;
map(
    . as $dot
    | .datetime | fromdate
    | (strftime("%H") | tonumber) as $hour # 0 through 23 |
    | (strftime("%w") | tonumber) as $weekday # 0 is Sunday, 6 is Saturday |
    | (strftime("%m") | tonumber) as $month # 1 through 12 |
    |
    {
        datetime: .,
        state: $dot.state,
        hour: $hour,
        weekday: $weekday,
        month: $month
    }
)

# | _fix_outliers

# | _adjust_usage

| _residential_general_use_one_meter_2rs1a # RESIDENTIAL GENERAL USE – ONE METER: 2RS1A
# | _residential_general_use_and_space_heat_one_meter_2rs6a # RESIDENTIAL GENERAL USE AND SPACE HEAT - ONE METER: 2RS6A 
# | _2rtou_with_net_metering_2rtoun # 2RTOU, With Net Metering 2RTOUN
# | _rtou2_with_net_metering_rtou2n # RESIDENTIAL TIME OF USE TWO-PERIOD

| _add_rate_fees
| _apply_rate

# | _analyze_by("month";(_analyze_by("hour";_cost_adder))) | _flatten_analyze

| map(.cost) | add | _add_monthly_cost as $calculated
| {
    actual: $actual,
    calculated: $calculated
}

# | (
#     (
#         _2rtou_with_net_metering_2rtoun
#         | _add_rate_fees
#         | _apply_rate
#         | map(.cost) | add | _add_monthly_cost
#     ) -
#     (
#         _adjust_usage
#         | _2rtou_with_net_metering_2rtoun
#         | _add_rate_fees
#         | _apply_rate
#         | map(.cost) | add | _add_monthly_cost
#     )
# )
# | {"Yearly Savings ($)": ., "Battery Size (kWh)": $batterySize}

About

William Floyd — Fri, 15 Dec 2023 00:00:00 +0000

I am William Floyd, this is my website.

I currently work as a Software Engineer at ModoPayments, working on our payments orchestration platform.

Ghetto NAS Part 1

William Floyd — Tue, 29 Aug 2023 00:00:00 +0000

This is an ongoing project to build a custom NAS on the most minimal budget possible.

Use Case

Storing a large (30TB+) amount of infrequently accessed data that must still be immediately accessible (primarily Jellyfin, Nextcloud), with some level of safety.

Some details about my use case:

There will be no external network access except via a single local client mounting the drive and sharing via ZeroTier
There will be very few clients total
Most data is replaceable, though inconveniently so (media may be reacquired / restored from backups)
Neither latency nor throughput are very important

Bill of Materials

Quantity	Item	Per Unit Cost	Notes
3	Dell Wyse 3030LT Thin Client	$11	eBay - Fairly common, though may run out eventually - other thin clients will no doubt appear
3	HGST 10TB He10 510	$80	Amazon / eBay - Very common, can pick these up any day
3	ORICO 3.5in to USB enclosure	$25	Amazon - Could use another, this is what I chose, does the job for me
5	Ethernet Cables	$2.5	Amazon - $12.50 / 5 pack - Or whatever you have lying around
1	8 Port Ethernet Switch	$13	Amazon - Or whatever you have lying around
0.5kg	PLA	$20	For the NAS enclosure

Rationale

In order of importance for my use case: Price > Redundancy > Performance

Hardware

Thin Client

You simply cannot beat a whole working Linux box for $11. With 2GB RAM, 4GB eMMC, 1 GbE, 1 USB 3 port, and a bundled power adapter, it does the bare minimum I need.

HDD

Similarly, used enterprise drives deliver an amazing value. For less than $9 / T B o r j u s t o v e r$ 10/TB with the enclosure, these drives are the cheapest possible way to get storage right now. By using external enclosures we can also upgrade to larger drives in future, with minimal effort. No shucking required!

I buy ones that have a 5 year warranty (spoiler - it’s worth having!).

Networking

1GbE is plenty enough for me, but if in future I need more speed, I can find a network switch with 10GbE uplink and scale horizontally a fair bit. For now, a cheap unmanaged GbE switch will do just fine.

UPS

Not 100% required, but the peace of mind in having the whole system on a UPS is worth it.

Software

Gluster

I am using Gluster to run my NAS cluster. This is in large part due to its very modest hardware requirements, especially memory. I can run my nodes with less than 50% memory utilization, and not fill my limited eMMC storage either. It is very easy to work with, and offers flexible redundancy configurations.

Configuration

I am using Gluster with a dispersed volume, using the native client on my main server to mount the volume. Dispersed lets me add clusters of bricks fairly easily, which suits my needs well.

Netdata

This lets me know if/when drives get full, lets me know drive temperature from SMART data, and will email me if any hosts go offline.

Experiences so far

I’ve been too busy to document the whole process, but I currently have a 2 x (2 + 1) array running (if I’d known I’d need 6 drives, I’d have done 1 x (4 + 2), but I didn’t know at first). Capacity is 60TB raw, 40TB usable.

HDD Failures

That 5 year warranty I mentioned? I’ve needed it twice so far - one drive died about 1 month in, and a second died 2 months in. To their credit, the vendor got me a return package label within one business day each time, and refunded me as soon as the return package arrived. For now, I continue to use these drives because the $/TB is so good, but in future I may upgrade to some larger drives in the same way to keep power costs down.

Power Draw

6 x HDDs + 6 x Thin Clients + Network Switch + 12V Power Supply, draws about 40W at the wall under regular load (serving files).

Topology

Not shown in feeds: a Mermaid diagram, drawn by JavaScript. View it on the site.

Smart Yogurt Maker Part 2

William Floyd — Wed, 16 Mar 2022 00:00:00 +0000

Last time we met, I got my rice cooker reporting temperature and roughly maintaining a set-point, producing my first batch of yogurt. Now follows a tale of Accidents, Averages, and Algorithms (sorta).

An accident has occurred

How prophetic… My last post said so truly that I needed to “ensure the outlet will turn off if the ESP8266 loses connection”. Of course, I couldn’t help myself and tried making a new batch before fixing this. So it came to be that 6 hours into a batch of yogurt, it was unceremoniously boiled for a full hour.

At first, all hope was lost - the mess of solidified dairy mocked me from the teflon lining of my dear rice cooker. As I forlornly drained the whey, I figured I’d give it a taste… After all, it had been boiled to death, it couldn’t possibly hurt me?

Rice Maker $\to$ Yogurt Maker $\to$ Cheese Maker?

I had in fact made cheese. Not at all intentionally, and not very well (it was still somewhat yogurt-sour), but it was undeniably cheese. A rough straining through a cheese cloth yielded something more passable as cheese, if only barely. My girlfriend confirmed it seems to be very similar to Paneer, though she declined to taste it. I ate only a small portion before deciding it wasn’t worth the effort…

Simmer down now!

Clearly, it was time to get my automation in order. It didn’t take much to ensure the automation would turn the heat off in case of connection loss, but the regular spikes in temperature were yet to be sorted.

Fortune Telling

The primary issue in using PID for this system is thermal lag. While my temperature sensor is accurate for its location, it cannot account for the temperature gradient in the contents itself. It would require a complete model of the system, including factors such as convection rate from the rice cooker housing, specific heat capacity, power draw, ambient temperature, volume of contents, initial temperature of contents, etc. All things considered, this is well outside of the scope of this project - this is meant to make yogurt, and I can do with much less.

Dirty Shortcut

What I have settled on is a simple hybrid between simple and PID-informed bang–bang control. My automation simply turns the heater on if the PID value (ranging between 0 and 1) is above 0.95. This is the case on startup, for example. If the value is between 0.6 and 0.95, it will turn it on, wait 750 milliseconds, turn back off, then delay any further action for the next 15 seconds. This ensures when attempting to hold a setpoint that the thermal lag in the system will catch up. These values are found purely by experimentation, but yield sub $1^{\circ} F$ variation easily.

PID Settings

Given some changes to the PID integration I am using, as well as further testing, these are now my PID settings:

\begin{array}{c} P & = 175 \\ I & = 0 \\ D & = 0 \end{array}

So really, it’s just the “P” in “PID” - I don’t mind if it works.

“Next time, on Rice Cooker Adventures”

Completed since last time:

ensure the outlet will turn off if the ESP8266 loses connection
PID tune the setup for the most common scenario (steady $107^{\circ} F$ with the lid shut)

Left to do now:

solder a perfboard version of the circuit
add a relay to control the cooker directly

Smart Yogurt Maker Part 1

William Floyd — Tue, 25 Jan 2022 00:00:00 +0000

A certain Rice Cooker has been languishing in a box for the last several years - now it is time to revisit it. It is a tale of Solder, Software, and Streptococcus thermophilus.

“Last time, on Rice Cooker Adventures”

This story picks up where I left off 3 years ago - the rice cooker has thermistors taped to the bottom of the pot with high temperature fiberglass tape. Specifically, I have 4 of the same thermistor in 2 parallel lines of 2 series resistors, like so.

This serves to roughly average the resistances, to account for possible hotspots on the pot, while only taking a single ADC pin.

However, time has not been kind to my memory, and the specific parameters of these thermistors are unknown, so first, it’s time to do some testing.

Side Quest: Thermistor Calibration

I have a thermocouple that I can use to verify the temperature of the pot with, so here is the setup: log the resistance through the thermistor bank at close to room temperature, rolling boil, and a middle temperature (during cooldown). These three sample points may then be used to find the parameters of the thermistor exactly.

From hazy memory (and counting parts in inventory), my suspicion was that I had used a $100 k Ω$ thermistor. Brand is unknown, but representative datasheets online cite a $β$ value of 3950 - let’s see how we compare.

Wait, what does $β$ mean?

For a more in depth read, look here, but in summary early studies proposed that for NTC thermistors, the temperature $T$ could be determined given a measured resistance $R$ , coefficient $β$ and reference resistance $R_{0}$ and temperature $T_{0}$ , like so:

\frac{1}{T} = \frac{1}{T_{0}} + \frac{1}{β} \ln \frac{R}{R_{0}}

While this may work for small temperature ranges (approximately $20^{\circ} C$ or so), we require a larger range, and so we are instead going to look at the…

Steinhart-Hart Equation

Briefly summarizing Wikipedia, the equation states:

\frac{1}{T} = A + B \cdot \ln {(R)} + C (\ln (R))^{3}

Which, when using 3 sample points, results in:

[\begin{array}{c} 1 & \ln R_{1} & \ln^{3} R_{1} \\ 1 & \ln R_{2} & \ln^{3} R_{2} \\ 1 & \ln R_{3} & \ln^{3} R_{3} \end{array}] [\begin{array}{c} A \\ B \\ C \end{array}] = [\begin{array}{c} \frac{1}{T_{1}} \\ \frac{1}{T_{2}} \\ \frac{1}{T_{3}} \end{array}]

In my case, I found the points of reference using a setup like so:

Do note that in order to reach a full boil (and $100^{\circ} C$ on the thermocouple), the lid of the rice cooker was attached.

While I could show the calculations, and they wouldn’t be difficult, I’ll be honest: I used an online calculator.

Using the measurements:

\begin{array}{c} R (Ω) & T (^{\circ} C) \\ 81 k & 33 \\ 21.3 k & 66 \\ 6.4 k & 100 \end{array}

Yielded

\begin{array}{c} A & = 1.032978070 \cdot 10^{- 3} \\ B & = 1.733021623 \cdot 10^{- 4} \\ C & = 1.902682261 \cdot 10^{- 7} \\ R_{(25^{\circ} C)} & = 117072.68 Ω \\ β_{calculated} & = 4202.76 K \end{array}

As we can see then, the actual $β$ value is larger than predicted ( $3950 K$ predicted vs. $4203 K$ actual), as is the reference resistance ( $100 k Ω$ predicted vs. $117 k Ω$ actual). In fact, neither of these are even in the commonly stated “ $\pm 1 %$ ” range. Good thing we’re calibrating, hey?

“Back to our regularly scheduled programming”

How to use this data then? Being short on time, I opted to take the easy way out, and use the wonderful ESPHome, as I already use Home Assistant. As it turns out, they already have us covered, with a NTC Sensor component already built in. This, coupled with a resistance and ADC sensor allow using a voltage divider to measure the resistance of the thermistor.

You know those calculations and equations we did above? Unnecessary, all of it. ESPHome supports doing Steinhart-Hart calculations itself, and only needs the three sample points.

In the meantime, a Wemos D1 Mini on a breadboard is ready to go (they’re cheap and cheerful). I wire it up with a $6.8 k Ω$ resistor (what I had handy), and away I went. The circuit looks something like this, where the two trailing wires connect to the thermistor bank:

To prevent confusion, here is my final working config, then I’ll explain.

sensor:
  - platform: ntc
    sensor: resistance_sensor
    calibration:
      - 81.0kOhm -> 33°C
      - 21.3kOhm -> 66°C
      - 6.4kOhm -> 100°C
    name: Rice Cooker Temperature
    unit_of_measurement: °C

  - platform: resistance
    id: resistance_sensor
    sensor: source_sensor
    configuration: DOWNSTREAM
    resistor: 6.96kOhm
    reference_voltage: 3.306V
    name: Resistance Sensor

  - platform: adc
    id: source_sensor
    pin: A0
    name: Voltage Sensor
    filters:
      - multiply: 3.2
    update_interval: 1s

Calibrate all the things

Series Resistor

Easy, measuring mine I found it to be $6.96 k Ω$ , certainly different from the $6.8 k Ω$ it’s meant to be. Throw that in the config.

Reference Voltage

This is also easy enough, measure the $3.3 V$ pin off the D1 mini. Mine came out to be $3.306 V$ , respectably close to rated! That goes in the config too.

Multiply Filter

As it turns out, the D1 mini has an internal voltage divider to convert the $3.3 V$ of the input voltage down to the $1 V$ the ADC is rated for. This is nice, as it protects the ADC, but it must be accounted for. But there’s a catch - on my board this wasn’t quite right - the voltage read was still slightly wrong. I could try manually tuning this, but there is an easier way. Instead, I measured the actual voltage ( $3.306 V$ ), and compared it to the voltage being read.

Say there was no error, then we would say $V_{actual} = V_{measured}$ , and if there is an error of a multiple difference between the two, we can say $V_{actual} = m \cdot V_{measured}$ . Thus, to find this multiplier, we say:

m = \frac{V_{actual}}{V_{measured}}

Using this method, I find my multiplier to be $3.2$ , not $3.3$ .

With that, the board is finally reading the correct voltage, the correct resistance, and the correct temperature.

“You’re so controlling!”

Given that I have Home Assistant, automating it becomes easy.

I’m currently using a Tasmota flashed smart outlet to control the power to the heating element of the rice cooker (for reference, it pulls up to $400 W$ at full load). In future, I would like to condense all functionality into a relay controlled directly by the ESP8266, to allow greater safety in case of network issues or even local PID tuning.

I am also using a PID controller for Home Assistant. It seems to work fine, I am just bad at PID tuning, so results may vary depending on hardware, amount of mass in the cooker, ambient temperatures, etc. That being said, a simple automation to turn the heater on when the PID response goes high and turn off when not high yields acceptable results.

From fiddling with the settings, I currently am using the following parameters:

\begin{array}{c} P & = 60 \\ I & = 5.6 \\ D & = 6.7 \end{array}

Further tuning is required, but this held well overnight - mostly…

I did change the set point from $105^{\circ} F$ to $107^{\circ} F$ halfway through, as well as tweaked some of the PID tuning. The concerning spike at 9:00 is marked by a momentary loss of connection with the ESP board - my automation failed to account for a loss of connection and so allowed the heat to continue rising. Thankfully, this was a short-lived event, but it is concerning nonetheless, and will require testing of my automation to prove that power loss tends towards a “safer” outcome. Do note that “safer” means no dead yogurt culture - the safety features of the rice cooker remain unchanged and functional.

The proof is in the… yogurt?

So how did it do? I am notoriously not picky when it comes to food, but I think it turned out great! Filling the rice cooker about halfway with whole milk, I heated it to $180^{\circ} F$ for a few minutes (I read I ought to go for longer…), cooled to about $110^{\circ} F$ , then added a liberal few spoonfuls (think 1/5 volume of milk) of plain Dannon yogurt. I let it proof in the rice cooker for 7.5 hours, with the lid sealed.

I put the whole rice cooker in the fridge to cool, with the lid sealed. Trying it later that same evening, I quite enjoyed it. I’ll start my next batch from this one, and hopefully it’ll be even better.

“Next time, on Rice Cooker Adventures”

Now that I have a temperature controllable rice-cooker, and the ability to remotely start, stop, and modify set points, I may try other foods. For now, I need to:

solder a perfboard version of the circuit
add a relay to control the cooker directly
ensure the outlet will turn off if the ESP8266 loses connection
PID tune the setup for the most common scenario (steady $107^{\circ} F$ with the lid shut)

First though, I’m gonna go eat some fresh yogurt…

$20 pen plotter

William Floyd — Sun, 26 Dec 2021 00:00:00 +0000

This was the final assignment for my second semester freshman engineering class (IEP 2). Budget given was $20, and my teammates and I decided a pen plotter would be a fun project. Here are some photos mostly of the final results

Home Automation at School

William Floyd — Wed, 21 Apr 2021 00:00:00 +0000

I love home automation - I’ve spent far longer writing automation routines and hacking together my own devices and programs than I have ever saved by doing so, and that’s perfectly fine. One unique aspect of my setup, however, is that I cannot control the network I must use - in my Uni dorm, I am not allowed to run my own router, and so all IoT devices must connect to the school wireless network. There abound dozens of Google Home devices, Chromecasts, and so on, all accessible on the same network - but not from the wired connection that my server/desktop uses.

Here then is my solution: MQTT everything I can. From my lights, to light sensor, to coffee maker to desktop software, I bounce it through a MQTT server hosted on a VPS. I use Home Assistant, so automatic discovery is easy on most things, especially ESPHome. In fact, I disabled direct Home Assistant connectivity entirely on these devices, which works well enough for me to live with.

But how to flash?

Given that my Home Assistant instance isn’t even on the same network as the IoT devices, how do I update the firmware? Using my laptop, I can connect to the wireless network that they reside on, and using IP address sensors reported by these devices, flash them directly without needing local discovery. In fact, I can easily automate this for myself using a couple scripts and a minimal number of hard-coded values:

hassio.sh

#!/bin/bash

export HASS_SERVER=https://<server_url>:443
export HASS_TOKEN='<HA_Token>'

hass-cli ${@}

exit

lamps.sh

#!/bin/bash

declare -A aa

aa["gosund_lb1_1.yaml"]="sensor.desk_lamp_ip_address"
aa["gosund_lb1_2.yaml"]="sensor.bed_lamp_ip_address"
aa["gosund_lb1_3.yaml"]="sensor.floor_lamp_girlfriend_ip_address"
aa["gosund_lb1_4.yaml"]="sensor.desk_lamp_girlfriend_ip_address"
aa["gosund_lb1_5.yaml"]="sensor.floor_lamp_ip_address"

__flash() {

    __config="${1}"
    __entity_name="${aa[${__config}]}"
    echo "Getting ${__config} IP..."

    __ip="$(
        ./hassio.sh -o yaml state get \
            "${__entity_name}" |
            grep -E '^ *state' | sed -e 's/.* //'
    )"

    echo "IP: ${__ip}"

    if [ "${__ip}" == 'unavailable' ]; then
        echo 'Ignoring...'
    else
        echo "Flashing..."
        ./esphome.sh "${__config}" run --upload-port="${__ip}"
    fi
    
    echo

}

if [ "${#}" -gt 0 ]; then
    until [ "${#}" == 0 ]; do
        __flash "${1}"
        shift
    done
else
    for __config in ${!aa[@]}; do
        __flash "${__config}"
    done
fi

exit

this allows me to mostly painlessly flash my devices, though truth be told there is little need.

Custom software

I developed for myself a tool in Golang to help tie more of my devices together. It is rather uncreatively/cryptically named ha-mqtt-iot - that is, “Home Assistant MQTT Internet of Things”. I may rename this some day, but why bother. It is similar to IOTLink (which is Windows only), and HASS Workstation Service - they are great projects, but this one is mine, even if it is poorly written.

The gist of the software is that most (all?) device types supported by Home Assistant may be implemented using a selection of user defined commands. The most prominent example in my case is to enable/disable dark mode on my desktop. I automate this according to ambient light in my room, to better match the aesthetic I want. Additionally, I can use it to turn my desktop monitor off, without resorting to using a relay outlet, and even change the color temperature of my system. The script I use for this looks like the following:

#!/bin/bash

__monitor_i2c='dev:/dev/i2c-3'
__monitor_dpms='0xd6'
__monitor_brightness='0x10'
__monitor_standby='4'
__monitor_off='5'
__monitor_on='1'

__unknown() {
    echo "Unknown ${1}"
}

f2i() {
    awk 'BEGIN{for (i=1; i<ARGC;i++)
   printf "%.0f\n", ARGV[i]}' "$@"
}

com="${1}"
arg="${2}"

case "${com}" in
"command")
    case "${arg}" in
    "ON")
        xset dpms force on
        ;;
    "OFF")
        (
            #xset dpms force off
            #sleep 0.5s
            #ddccontrol -r "${__monitor_dpms}" -w "${__monitor_standby}" "${__monitor_i2c}" -f
            #sleep 2s
            ddccontrol -r "${__monitor_dpms}" -w "${__monitor_off}" "${__monitor_i2c}" -f
        ) &
        ;;
    *)
        __unknown "${arg}"
        ;;
    esac
    ;;
"command-state")
    echo -n "$(xset q | grep 'Monitor is' | sed -e 's/.* //' | tr '[:lower:]' '[:upper:]')"
    ;;
"color-temp")
    v="$(f2i "$(bc -l <<<"1000000/${arg}")")"
    ./scripts/run-in-user-session.sh gsettings set org.gnome.settings-daemon.plugins.color night-light-temperature "${v}"
    ;;
"color-temp-state")
    v="$(./scripts/run-in-user-session.sh gsettings get org.gnome.settings-daemon.plugins.color night-light-temperature)"
    echo -n "$(f2i "$(bc -l <<<"1000000/${v/* /}")")"
    ;;
"brightness")
    ddccontrol -r "${__monitor_brightness}" "${__monitor_i2c}" -w "${arg}"
    ;;
"brightness-state")
    echo -n "$(ddccontrol 2>/dev/null -r "${__monitor_brightness}" "${__monitor_i2c}" | tail -n 1 | grep -o '/[0-9]*/100' | sed -e 's|^/||' -e 's|/.*||')"
    ;;

*)
    __unknown "root command ${com}"
    ;;
esac

exit

Note that the display doesn’t respond to being turned back on, so this is somewhat incomplete, but it’s good enough for my needs. The corresponding portion of the config for ha-mqtt-iot looks like the following:

"lights": [
    {
        "info": {
            "name": "Desktop Monitor",
            "id": "monitor"
        },
        "command": [
            "./scripts/monitor.sh",
            "command"
        ],
        "command_state": [
            "./scripts/monitor.sh",
            "command-state"
        ],
        "command_color_temp": [
            "./scripts/monitor.sh",
            "color-temp"
        ],
        "command_color_temp_state": [
            "./scripts/monitor.sh",
            "color-temp-state"
        ],
        "command_brightness": [
            "./scripts/monitor.sh",
            "brightness"
        ],
        "command_brightness_state": [
            "./scripts/monitor.sh",
            "brightness-state"
        ],
        "brightness_scale": 100,
        "update_interval": 5
    }
]

Pretty simple. This makes custom system sensors trivial. For example, to show my system IP, I use the following:

"sensors": [
    {
        "info": {
            "name": "IP Address Desktop Solus",
            "id": "ip-address-desktop-solus",
            "icon": "mdi:ip-network"
        },
        "command_state": [
            "/bin/bash",
            "-c",
            "ip -j address show eno1 | jq -r '.[0].addr_info[0].local'"
        ],
        "update_interval": 10
    }
]

Some common use cases are built in as well. Currently, this includes laptop displays (as lights) and batteries (as sensors), as well as Crypto prices (through the CoinGecko Golang library). These are really easy to call. An exhaustive example is quite short:

"builtin": {
    "prefix": "Name Prefix ",
    "backlight": {
        "enable": true,
        "temperature": false,
        "range": {
            "minimum": 0.025,
            "maximum": 0.95
        }
    },
    "battery": {
        "enable": true
    },
    "crypto": [
        {
            "coin_name": "dogecoin",
            "currency_name": "usd",
            "update_interval": 1,
            "icon": "mdi:currency-usd"
        }
    ]
}

This lets me tailor my setup to each machine I’m using, while still enjoying the benefits of Home Assistant MQTT Discovery. The primary limitation at present is the inability to signal to ha-mqtt-iot from another process - it can only poll for changes. This will be addressed one day, when it is important for my own needs.

How to host?

But the question is now, how do I access my HomeAssistant instance if it’s also hosted at school? I most certainly don’t have a public IP, so in comes AutoSSH. I’m not sure which is the best one at this stage, but refer to this repo and check the various forks of the parent project.

I have configured on my VPS a docker image that accepts reverse SSH tunnelling, authorized only to the key of the HA addon. From my docker-compose.yml:

homeassistant:
    image: "docker.io/panubo/sshd"
    container_name: homeassistant
    environment:
        - TCP_FORWARDING=true
        - GATEWAY_PORTS=true
        - SSH_ENABLE_ROOT=true
        - DISABLE_SFTP=true
    volumes:
        - "./hassio/authorized_keys:/root/.ssh/authorized_keys:ro"
        - ./docker-config/hassio/data/:/data
        - ./docker-config/hassio/keys/:/etc/ssh/keys
    ports:
        - "<MY_PORT>:22"
    restart: unless-stopped
    hostname: "homeassistant"

This is then reverse proxied to using Caddy, to expose the website on a subdomain of a website. From my Caddyfile:

<MY_SUBDOMAIN>.{$MY_DOMAIN} {
    reverse_proxy homeassistant:8123
    encode gzip
}

Pretty simple, but not without some hiccups now and again - I occasionally have to restart the sshd docker on my VPS if something goes wrong with HomeAssistant.

Bonus: Android tie in

I use Sleep As Android to track my sleeping patterns, and as my alarm clock. Using Tasker, I can run an action when I begin sleep tracking, which (using a HomeAssistant plugin for Tasker) can call a script on my HomeAssistant instance to turn off my lights (only if I’m home, of course). Similarly, it turns on my bedhead light when my alarm goes off in the morning, and could optionally make me coffee…

Analysis of a 2005 Nissan Altima Remote Keyfob

William Floyd — Fri, 08 Jan 2021 00:00:00 +0000

I got my first car a few months ago - a decidedly beat up North American 2005 Nissan Altima. It came with a remote keyfob, but something was shorting internally, and the battery would drain within a matter of hours. I ordered a two-pack of aftermarket fobs from Amazon, paired them no problem, and that was that.

After a while though, I got to thinking, how might I connect my car to my Home Assistant system? I am reluctant to tap into the CAN bus directly, so instead I am going to investigate utilizing the same method as my remote fob. Specifically, I want to program my own “fob” using an Arduino and a RF transmitter (spoiler: I cannot, at least for now).

The Hardware

While I may analyze the original fob at some point, for the time being I shall work from the aftermarket fob. Opening the casing reveals a rather simple circuit (front, back) - battery contacts, four buttons, a smattering of passives, an oscillator, the main control chip, 8 test points, and an antenna trace running around the perimeter of the PCB.

Rather unhelpfully, the control IC is unmarked, so either I’ll need to wait until I can look at the original fob to find any clues, or try to find what I need online.

As a note, it should be easy enough for me to replace the buttons with an electrical solution controlled by an ESP8266 or similar - I may end up doing this some day. For the time being, however, I have nothing to glean from the hardware in my fob.

Research

A quick look at the Amazon listing for my fob already tells me what models of fobs it replaces: KBRASTU15, CWTWB1U758, and CWTWB1U821. Matching other listings, my specific model is the KBRASTU15. This is actually an FCC ID, so that’s my next lead.

This fccid.io listing is rather useful, providing original FCC images and documents. Unfortunately, the internal images are low enough resolution to prevent reading the IC silkscreen. Interestingly, the aftermarket fob has a small fraction of the part count of the original fob, while still functioning perfectly. The most useful available document is the Technical Description. This is where we start to get somewhere.

In essence, the fob emits a wakeup sequence ‘A’, followed by sequence ‘B’ repeated as long as the button is pressed. The structure of these sequences is also described - sufficient to say, it uses a rolling code system. Importantly for my analysis, the encoding method of the signal is given (though interestingly their diagram annotations seem to be incorrect).

With this information in hand, I ordered a cheap RTL-SDR (a Nooelec NESDR Mini 2+).

SDR Analysis

The radio, being a North American model, operates at 315MHz. Using Universal Radio Hacker, I began capturing examples of locking and unlocking signals. Using the software is outside the scope of this post, but I found some YouTube videos by the software author that proved to be useful - it’s not hard to learn to use.

Ultimately, I ended up trimming down 4 ‘B’ sequences each of locking and unlocking (that is, different presses, not repeats during the same press). I also opted not to include the preamble and header of these ‘B’ sequences, as these are the same for any fob and include no actual information.

The resulting signals varied in length and followed no apparent repeating format - this is where the encoding scheme described in the FCC documents comes in. Fortunately, URH allows specifying and chaining encoding schemes, as well as calling external software to decode the data. As I am not dealing with a large amount of data, I wrote a quick bash script to decode the data:

#!/bin/bash

lc=''

while IFS= read -r -n1 c; do

        if [ "${c}" == '' ]; then
                break
        fi

        if [ "${c}" != "${lc}" ] && [ "${lc}" != '' ]; then
                echo -n "${lc}" # Deal with the last character
                lc="${c}"
        elif [ "${lc}" == "${c}" ]; then
                if [ "${c}" == '0' ]; then
                        echo -n '1'
                else
                        echo -n '0'
                fi
                lc=''
        elif [ "${lc}" == '' ]; then
                lc="${c}"
        fi

done <<<"${1}"

exit

This does the trick, and all the data has a uniform length of 67 bits. While the document specifies 66 bits, perhaps the aftermarket keyfob adds an extra bit. In any case, this extra trailing bit is always 0.

Finally we have something like this:

The top four entries are when locking, the bottom four when unlocking. The highlighted segments from left to right, as per the technical document, are:

encrypt
serial number
function code
verify code

It can easily be seen how the intent of the button press is not encrypted, but is part of the function code near the end. Also, the serial is correctly defined, as it does not change at any point. I am not sure what the verify bits are present for - perhaps if an error occurs the encoding scheme would result in a non 0 result (though this is purely speculation).

What we know now is that the keyfob does indeed use a rolling code algorithm, though not which one (web searches do not seem to specify).

More Research

I do not have access to the original keyfob at this time, so some image searching may be in order. Several leads appeared, but the main one was this:

None of the text appears especially legible, so the first thing I can do is compare the logo on the OEM fob to existing rolling code chip manufacturers. As it happens, Microchip is the company in question, producing the widely documented KeeLoq product line (leaked spec sheet). Looking at their product lines, it becomes clear that the original fob uses the HCS361 chip. My aftermarket chips must be using a knockoff or unmarked version of this.

Hurray, we know what chip we’re up against, what algorithm it uses, and how to get the information of an existing fob. What we do not know is how to program our own chip (or software equivalent). This is where things get a lot more involved than I have the expertise to do.

What would be required would be the Nissan manufacturer code for this series of fobs. Try as I might, these do not seem to have leaked online, nor have any other manufacturers as far as I can tell. Speculating, perhaps those that have access to these codes either keep them to themselves for ethical reasons, or because there is financial incentive to keeping this information private. Clearly there are Chinese manufacturers who know what this key is, because it would have been required in order to program this fob.

Key Problem

So how might I find this manufacturer key? Curiously, I cannot even find reference to Nissan ever using KeeLoq, so that does not bode well. Trawling the recent NA Nissan git leak doesn’t reveal anything related to such a key. The consensus seems to be that a power analysis of the receiver is required to determine the manufacturer key. While apparently not difficult for the researchers, it is beyond my means at this stage. I am certainly not the first one to try this.

The way forward would seem to buy a spare ECU for my vehicle and to use a ChipWhisperer and attempt to learn how to do power analysis. I don’t have the time or resources (or frankly the need) to do that for now, so I think this is where my journey ends. I’ll update this if ever I can get my hands on the manufacturer key and do anything more with this.

DIY 'Smart' Coffee Maker

William Floyd — Sun, 05 Jul 2020 00:00:00 +0000

I have recently (last couple months) been getting into home automation, specifically using Home Assistant. I also wanted to get a coffee maker that used K-Cups. Thus the goal was born - to get a coffee maker working with Home Assistant, such that I could automate it to fit my morning routine.

I first searched for new coffee makers, looking to see what the easiest ones would be to hack into and automate. I got so far as adding a sleek new brand name model to my Amazon cart, but thankfully reason and frugality prevailed! Instead, I opted to swing by the local Goodwill the next day, to see what secondhand options might be available. As luck would have it, I found a no-frills, single button coffee maker for the princely sum of $4, and I was on my merry way.

Later that evening, opening the black box up (thankfully I had the correct triangular screwdriver head), I found very easily hackable innards.

The control board (front, back) is very simple - two buttons in parallel, an LED, and a couple resistors. I’m not quite sure what the logic is upstream (it un/latches on each press), but the buttons short to ground, and this is something I can work with. The control board has wires for ground, signal, and 5V, in that order.

The ordeal of hacking into the thing, and my missteps trying to do so, is rather boring really, but it produced this:

The brains I added was a Wemos D1 Mini (thanks, Aliexpress!), with a 3.3V<->5V converter between to sense when the coffee maker is on, and also turn on a mosfet to short the button to ground. I initially had issues with the coffee maker turning on whenever I plugged the D1 Mini in, but a pulldown resistor solved this.

I also opted for an external USB plug - this allows me to possibly reflash this without opening the enclosure, and also meant I did not need to splice into the mains line inside, which I was reluctant to do.

I initially flashed and programmed the coffee maker with Tasmota, but almost immediately opted to use ESPHome instead. I like Tasmota and ESPHome, I was just able to get more fine control of the setup using ESPHome in this instance, especially concerning the button logic.

The (important) code is as follows:

binary_sensor:
  - platform: gpio
    device_class: power
    internal: true
    id: power_on
    pin: D2
    
switch:
  - platform: gpio
    pin: D1
    id: relay
    restore_mode: ALWAYS_OFF
  - platform: template
    name: "Coffee Maker"
    icon: "mdi:coffee-maker"
    lambda: |-
      if (id(power_on).state) {
        return true;
      } else {
        return false;
      }
    turn_on_action:
    - switch.turn_on: relay
    - delay: 50ms
    - switch.turn_off: relay
    turn_off_action:
    - switch.turn_on: relay
    - delay: 50ms
    - switch.turn_off: relay

The beauty of the way in which I can detect signals and simulate a button press is that ESPHome senses when I use the physical button, and updates accordingly. I can use the coffee maker absolutely as normal, the ‘Smart’ part of it is optional.

The rest of my setup using this coffee maker is somewhat outside the scope of this post, but it includes using Tasker on my phone to run a ‘Wakeup’ automation on Home Assistant, which will - along with turning on my bed-head lamp, and switching my monitor on before putting it to sleep - check if I turned on a ‘Morning Coffee’ user input. If I remembered to prep my coffee the night before, and turned on the switch, I will wake up to a hot cup of coffee (or, the sound of a coffee-in-progress). I can also command Alexa and Google Assistant to do my bidding and make me a coffee. At least someone does what I want…

Truly, we are living in the future.

midiMixer - A simple Arduino powered MIDI potentiometer bank

William Floyd — Wed, 06 May 2020 00:00:00 +0000

This is nothing too special really - I wanted to try my hand at designing a custom PCB and assembling a device from scratch. GitHub for this project.

This story actually starts a while ago (perhaps 2015), when I got the desire to build a DJ control panel (with no knowledge of how to get into it). This never really happened, but it did lead me to get to know some DAW software, and experiment with it off and on. But in the back of my head I still wanted to build some hardware to help with this.

This eventually came about in the summer of 2019, when I finally hand-wired a simple 8 knob control surface. Truth be told, I actually built two. The first one had some issues (I wonder why…?), and would sporadically jitter values. The second one worked correctly, having been wired much more carefully.

Still, it was lacking professionalism, polish, and most importantly, robustness. The sandwich of 22AWG wire, cardstock and PCB was less than sturdy, so I promptly packed it away and never used it. Around this time also, I tried very briefly to learn how to use KiCAD, but didn’t get far at all. This would come to change, however, in April of 2020.

Fast forward close to a year, and here we are - toilet paper is the currency of the quickly collapsing Covid19 crazed world, and I’m back with my grandparents for the remainder of my semester (which, as of writing, is all but concluded). Finding my lack of a social life leaving me with a little bit more free time, and being home yielding a lot more distractions, I learned the basics of KiCAD from YouTube, and set out to design a simple PCB. My old MIDI controller project came to mind, so I set myself to it, and within a few days had my first Gerber files sent off to JLCPCB. In reality, I should have waited a bit and done a couple more design iterations before committing, but I guess $7 is the price of impatience.

So it was that my crisp PCBs showed up 3 weeks later - I had one assembled within the hour, and my old code dusted off and flashed within two hours. It’s really as simple as it gets - a Pro Micro hooked up to a 4051 analogue multiplexer, in turn hooked up to 8 potentiometers.

The potentiometers I had on hand ended up being a bit different than the footprints I had used, so I ended up having to clip the mounting tabs off of them. This, as one might imagine, made them extraordinarily flimsy, so I designed and printed a couple clamps to align and hold solid the potentiometers. These also have the added benefit of keeping the soldered pins off of whatever surface the mixer is on. Should the correct potentiometers be used, these would not be needed, though some feet or a case for the PCB would still be best.

This being my first PCB design, I am quite happy it worked out so well - I am studying for Mechanical Engineering, not Electrical… However, I see room for improvement. The potentiometers are too close together, and the rows are too far apart. The knobs barely clear the Arduino, and the Arduino is mounted far higher off the PCB than I would like. I see myself making a v2 in due course - perhaps next time I will try JLCPCB’s SMT assembly? I imagine adding indicator LEDs, improving the layout, and adding a button to switch between banks would make this much more useful.

Clickbait - The Automatic Fishing Caster

William Floyd — Fri, 13 Mar 2020 00:00:00 +0000

This project was completed in the Fall Semester of 2019, for my EPM class at LeTourneau University.

The mandate given to us was as follows:

Create a device to launch a fishing weight. It must:

be capable of selectively launching 20 and 40 feet
weigh less than 5 pounds
cost less than $100
be battery powered
be capable of firing 30 times over the course of 1 hour
(for extra credit) be wirelessly controlled

And so it was that we began brainstorming.

The idea was fairly simple - use two smooth rods and a leadscrew to pull a carriage against springs that would then somehow fire the fishing weight. The choice of this mechanism was, for the most part, due to availability of parts. I already owned the necessary rods and bearings, and springs were readily available.

Soon, we had some basic 3D printed parts in order - for it was 3D printing that was most accessible to us, and allowed largely unattended manufacturing while classes continued. Despite this early start of progress, however, the ever present tendency toward procrastination crept in. The usual array of excuses were made by all, and little actual work was accomplished.

So it was that we inched ever closer to the deadline. Past the basic idea of our launch mechanism, all else was yet to be decided. A few calculations were half-heartedly done (kinetic energy, spring constants, etc.), but we all knew we didn’t know enough to make any meaningful decisions yet. We had some very much unfounded fears concerning battery power, and not enough fears concerning torque - we believed we would be using NEMA 17 motors to tension the leadscrew. Needless to say, this did not turn out to be realistic. Nor was it realistic for us to budget a baitcasting reel instead of a much cheaper spincaster.

Eventually, however, some progress was made - progress in something of a wrong direction, but progress nonetheless. As the shape of our ungainly creation began to emerge, it was clear progress needed to be made quickly. Once we coupled our stepper motor to the leadscrew and power tested the unit, it was also clear that change was in order.

So a decision was made: a cheap cordless drill would be pilfered for a battery, motor and chuck. An H-Bridge would need to be bought for the drill motor (I tried to make one, but the magic smoke was released…), and a latching system needed to be made.

The next week or so resulted in the beginnings of a working launch mechanism. With the aid of a rubber band, a latching track was put under the rods and springs, such that the motor would tension the launching carriage, then return to the front of the mechanism to allow the eventual release of the catch using a servo. Soon after this, the motor was appropriately mounted to the frame, with a carefully fit 3D printed socket. It was, it seemed, all coming together.

▶ Watch the video

Finally, some success came to us. The motor was successfully power tested, along with the release of the launch mechanism. The end was near, or so it seemed.

Following soon after was the reel being mounted to the device also. Attempts were made to use a small DC motor, but with no luck. The lateral pressure of our motor acting through a belt meant friction was too much for our motor. Without time enough to reposition it, another solution had to be made.

Remaining at this point was a means of reeling in the line, a means of depressing the button on the rear of the reel, a limit switch at the front of machine, and the connecting electronics that would drive the entire contraption.

As it turned out, the very same stepper motor that had once tried to turn our leadscrew now instead would be pressed into service reeling in the line. Overkill, to be sure, and its power most wastefully used, it did indeed reliably turn our reel, albeit slowly. The use of such a large pulley was a matter of making it fit the belt size that I had available. The next size down was too small, so the pulley had to be enlarged. This saved the most time 3D printing as compared to a smaller pulley with other parts changed.

Finally, the mechanics of the project were completed, with each component individually power tested, though not yet in tandem with one another. As all of this was coming about, I had also begun getting the electronics in order. A limit switch was installed to keep the motor from destroying the machine. When this switch invariably failed to activate, it resulted in the destruction of the front piece of the machine - at that stage more of a nuisance than a disaster. Spares were kept by, and small changes were made so the switch could be mounted elsewhere in a more reliable fashion.

By way of a multi-button garage door opener, and a 433MHz receiver, wireless control could be granted to an Arduino for little money. A cheap buck-converter was added in order to run the Arduino and wireless setup, as well as the stepper motor driver. Thankfully, the voltage of the battery meant the stepper motor could be powered from the battery directly (18V being safely within the rated 12-24V range)

▶ Watch the video

In the above video, the finished product may be seen.

Finally, presentation day came. Not without some small failings, we presented, with acceptable results. The largest remaining issue was the fact that the leadscrew would slip the grasp of the chuck after several cycles of the machine. While we corrected by hand for the presentation (by re-chucking the leadscrew after each launch), we knew we needed to fix the slippage for good. After all, the final time-lapse of the project would not do with such continual, and blatant, hand-holding.

The solution, it turned out, was free: the end of the leadscrew was notched and tapered on a lathe, allowing the drill chuck to get a firm hold on the metal.

▶ Watch the video

With this solved, the final time-lapse was completed, and a collective sigh of relief was given.

Much was learned - how to more properly budget for a project, how to work with a team (something I need to improve on), and how to manage time well. All things considered, I found it to be a success - I did not pull an all-nighter for my own project, finishing with a few days to spare (no small thanks to an extended deadline), nor did I stress myself beyond all reason. Overall then, I enjoyed the experience of finally bringing a project to completion - this is something I do not often do, try as I might. The result, though useless for my own purposes, is something I am proud of, having taught me many things through its completion.

The State of Now

William Floyd — Wed, 11 Mar 2020 00:00:00 +0000

Perhaps it is the slow pace of work over my spring break, or the ever mounting pressure to build a portfolio of content before I graduate, but it came to pass that I recently finished configuring this server. Truth be told, I merely transitioned to a different VPS provider, as I had lost SSH access to my prior server, and did not trust the host to begin with. But, whatever the case, here I am - as sporadic as ever, though perhaps a little older.

As such, perhaps some content should fill this - as of the moment - rather empty website.

Since my first posts here concerning a certain ill-fated rice cooker (a project that I admit to never having brought to fruition), both much and not much has happened. I am halfway through the 4th semester of my Mechanical Engineering degree at LeTourneau University, am happily in a relationship with a fellow engineering student (though I may not name her), and have become - at least somewhat - less of a pain to those around me. Yet I feel that though I have learnt much over what is soon to be 2 years of education, I have at the same time also done nothing of interest. The closest I have come yet are perhaps the engineering projects of 2nd and 3rd semesters, though I shudder at the memory.

My first semester passed - in retrospect - rather peacefully, but Spring of 2019 brought with it IEP 2, and Fall brought EPM. I feel then that the projects that came out of these classes should be given a proper writeup, so as to flesh out the content of this site.

Dumb Rice Cooker Testing

William Floyd — Mon, 02 Jul 2018 00:00:00 +0000

See my ongoing Smart Rice Cooker conversion post for my attempt at making the rice cooker smarter.

Test Cookings

2018-07-02.1

I don’t have much rice with me (Basmati, in this case), and I’m away from home for the week, so I only used a single scoop of rice. I believe the provided cup is 100ml (it is marked such that I assume this, and the volume appears correct), so my volume of rice was about 90ml and water 140ml. I set the rice cooker going and left it alone for 14 minutes or so, expecting to return to watch it finish. To my surprise, however, it was already done, dutifully warming my rice.

Not having added any salt or other things to make it more interesting, it was plain, but not unpleasant. Next time, I will try a fraction more water and adding some salt as I should have. Promising results though, as I have never cooked rice before, in a pot or otherwise.

2018-07-03.1

Tried again, 100ml of rice, slightly too much water (160ml), salted this time. Appeared slightly too wet, but perfectly tolerable. Put it on to warm for a minute or so longer, then carried it off. By the time I had gotten some chili to go with it, it was just fine. Very much met expectations (good), very much worthwhile in my mind.

Might try a slight bit more salt next time, but all seems well.

Smart Rice Cooker Conversion

William Floyd — Mon, 18 Jun 2018 00:00:00 +0000

This project has been stagnant for some time now. I’ll probably end up using 3D printer code and a G-Code translator to do what I want in the end.

The “why” isn’t important (read “doesn’t exist”), but I want to take a regular old “dumb” rice cooker, and convert it into a “smart” rice cooker.

So, a whim and a few minutes on Aliexpress later, my pint sized rice cooker and an assortment of electronics are on the slow boat from China.

Having no real clue what I’m doing, here’s what I’ve ordered thus far:

Main list of things:

Rice cooker ( $26.40 +$ 3.16 S&H - The star of the show.)
Thermistors ($2.97 - 100pcs, for temperature readings)
NodeMCU ($2.47 - To, hopefully, allow me to WiFi control the whole thing)
Relay ($0.76 - To switch power on and off.)
ESP8266 Relay ( $2.91 +$ 0.14 S&H - Alternative to the NodeMCU + Relay pairing I’m planning on using)

While I have dabbled with Arduino before, this is a far more ambitious project than I have yet done. Ideally, I will have the rice cooker serve an API, which my Orange Pi will bounce requests across to as dictated by a frontend hosted on the same machine (this resolves CORS, while still allowing options to control the rice cooker by other devices).

For now though, I am away from home, working, so I can’t do anything on this for at least another 2 weeks, probably 3. Many of my parts have arrived (rice cooker, thermistors, relay, and NodeMCU), so once I get home, I can hit the ground running.

Terminology

Directive

A directive is an instruction for the rice cooker. User facing directives have, at most, a single input.

Internally, directives chain each other with conditions to achieve their given goal.

Thoughts

Having received shipment of the rice cooker, and beginning to use it, I realize there’s nothing about it that makes it fit only for rice, if I can control the heating myself. Everything about it is like any other hot pot, it’s just tuned for cooking rice.

Thus, once I convert the thing to being smart, I will be able to use it for, say, ramen, or any number of other things. The possibility of steaming things only furthers this goal.

Recipe Submission

Wrap the JSON as an object so that flags and things can be passed along without modifying the recipe.

Recipe IDs are handled internally by the cooker, as a hash of the contents.

When submitting a recipe, the server will return the ID of the recipe, so the client may immediately attempt to start a recipe, but flags such as delete-on-complete make sure that recipes may be temporary in nature and not stay in history (parametric recipes are a prime example)

Recipes

Recipes should be submitted as simple JSON structures. These may be linted prior to submission, and will contain no logic. All calculations are to be done when producing the recipe. This is done so as to ease the technical burden on the cooker.

Tuning

Heating should be tuned through the frontend. A wattage value may be submitted as a rough guide.

Otherwise, say how much water you put in, it will heat up until a threshold temperature is reached. Wait until the heat dropping is stable (thermal lag needs to be gotten rid of), then read temperature. Given this information, the cooker may then calculate the heating rate of the hot pot, as the thermal capacity of the water is known, and the thermal lag is mostly taken care of.

This information may be used for ballpark estimates for cooking duration, when accounting for the time it takes to reach temperature. Examples of use might be scheduling a recipe in time to get home.

However, more useful will be using prior cooking runs to average the cooking times.

Heating / Cooling

Active cooling is not something worth designing for, as it is not an issue to begin with.

Coding

Internally, a temperature kill switch should be enacted, which will kick in if the temperature gets too high.

API

I tentatively (with no real experience designing them) plan on my API being something like the following.

/cookbook/add - Add a recipe
/cookbook/delete - Delete a recipe
/cookbook/list - List known recipes
/action/kill - Kill any current running recipe
/action/start - Start a recipe
/action/schedule/add - Schedule a recipe
/action/schedule/delete Delete a scheduled recipe
/action/schedule/list - List scheduled recipes
/sensor/temperature - Returns temperature
/settings/time/set - Set/read date and time

Cooking recipe definition specifications

They may be categorized as follows:

Primary Directives

Heat
Sleep

Secondary Directives

Lighting Change
Lighting Kill
Temperature Change
Temperature Hold
Temperature Kill

Definitions

These are defined as such:

Sleep

Wait for a given duration. No need to expose to recipes directly, see Temperature Kill, Temperature Change, and Temperature Hold instead. Will keep things like LED lights going still.

Heat

Takes a bool, turns the heating element on or off. Non-blocking so that conditions may be checked without unnecessarily cycling the relay. Dangerous to expose to recipes directly, see Temperature Kill

All Kill

Mostly to be used as a soft kill switch, invoked from a physical button or via API. Proxy to Temperature Kill and Lighting Kill, then exits any current recipe.

Lighting Change

Change lighting mode.

Lighting Kill

Kill any running lighting mode.

Temperature Change

Heat/cool to given temperature. This should just be a proxy directive to Heat and Sleep. That is, Heat if under temp, Sleep if over temp. As heating during cooking tends to be a matter of getting to temperature as quickly as possible, there is no need for a duration setting.