LLM on William Floyd (high-quality images)

Notes on LLMs - Part 2

William Floyd — Fri, 06 Nov 2026 00:00:00 +0000

Some notes on how I’ve found the most use out of LLMs. The specifics of the tools honestly change so fast, I won’t get into the details too much, but here’s a look at some themes I’ve found. Note that some of these principles are likely helpful even without LLMs, but are especially important with them.

TL;DR

Know your actual goal, and when “good enough” is all it needs.
Get something working first, and make the feedback loop as fast as you can so each iteration runs on its own.
Turn anything you repeat into a script: scripts are inspectable, reusable, and outlast cheap tokens.
Do homework on prior art and primary sources before running an LLM on a large or opaque task.
Reshape a task into a common, well-trodden form, so the novel work is small and contained.
Get both ends working, then use the LLM to fill in the step between them.
With profiling and low stakes, LLMs are good for trying out speedups.
LLMs are a hammer; not everything is a nail.
Pin down how you’ll measure features, performance and correctness, early.

Keep the “Main Thing” the Main Thing

Know what you’re looking to achieve when you set out on a project.

This depends heavily on the criticality of what’s being done, but know when “it ain’t stupid if it works” applies. For example, with my work in Part 1 getting a phone and then a car’s head unit running Linux, the goal was just that - getting Linux running.

Non-goals included:

sharing every random script used
documenting every step
doing it “right”
mainlining modifications

This sounds terrible, perhaps, but bear with me. The goal is to get mainline-ish Linux running on these devices. No-one else has done this, to my knowledge - not because it’s impossible or even hard for the right person (it is hard for me), but because I happen to be the first attempting it.

I assure you I had rather be the first man here than the second man in Rome.

— Julius Caesar (Wikiquote)

Put another way, if you can’t be the best, at least be first.

This is not, however, meant to disparage taking the time to do good work. On the contrary I value good work very much, and hope to do as much of it as I can. But sometimes what I need is “good enough” work to achieve a goal.

LLMs are a tool used to achieve this goal for me. Were my goal to simply write lines of code, there are myriad ways of fulfilling that goal, but rest assured I would not achieve the goal of booting mainline-ish Linux as quickly as I did with LLMs.

Make It Work, Make It Right, Make It Fast

Make it work, make it right, make it fast.

— attributed to Kent Beck (c2 wiki)

LLMs are fairly good at the first (and sometimes the third, see more here) part of this phrase. Run an LLM in a loop against a clear goal, and you can often brute-force something working, however hacky it might be.

Again, this may sound terrible, but I find it much easier to have a working end-point to work backwards from, than to do everything “right” from scratch. This is especially true of things like embedded Linux bringup, where simply getting a booting system with remote access is so much more helpful than any amount of “doing it right” blindly could ever yield. Once you get it working, it’s much easier to get it working right.

When using LLMs it is especially important to close the feedback loop as quickly as possible, so the tool can iterate automatically. For example, in Part 1, I ensured there was a script that automated as much of the USB-to-boot cycle as possible. See also xkcd 1205, “Is It Worth the Time?”. Which leads me to my next point…

Produce Durable Goods

Make scripts for everything you do more than twice. LLMs have a tendency to produce inscrutable one-liners for routine tasks, so I find it helpful to step in to ensure durable scripts are made instead, especially if I know I’m going to do the task often. These scripts don’t need to be especially good, but the fact they’re at least written down in one place and inspectable is critical. They can be committed, reviewed, and tweaked as time goes on. These scripts, once sufficiently worked through, serve to reduce the cognitive load on humans and context usage of LLMs, which then simply run a small command to do a rote task instead of reinventing the wheel.

These cobbled together scripts may be a rat’s nest as a whole, but if they work, they inch you closer to your goal. Once that goal is achieved, it is at least repeatable. And once it’s repeatable, it can be made more “right” (see above).

These “durable goods” will outlive the cheap LLM tokens. Once the true cost of operating LLMs at a loss funded by gobs of VC money is brought to bear, we will still have these outputs. Ultimately, the less you depend on LLMs being cheap, the better. And if they do remain cheap (or get cheaper), then all the better I guess…

Prior Art and Primary Sources

It is useful to survey prior art and primary sources before just setting LLMs running on a given task. For example, in Part 1, tracking down the somewhat obscure Mobis open source portal got me a vendor kernel tree. Perhaps an LLM would have found it eventually (and I’d still have had to download it myself), but by doing at least a modicum of up-front homework, a lot of time can be saved. In this instance, the LLMs could then derive vendor changes for the SoC in question and port them to a new mainline branch, something that would take me far, far longer.

Intermediate Formats and Task Normalization

Massage your tasks into something that is like something that is already commonly done. Once a common intermediate format is achieved, you will have a much easier time of making progress. This is especially true when reverse engineering, but may still prove useful in other projects. For example, converting an obtuse API spec from a 500-page PDF into an OpenAPI spec that is 95% complete with the last 5% captured in prose descriptions yielded better results with LLMs than just using the PDF directly.

I might call this “task normalization”: breaking a project down into the most idiomatic steps such that LLMs need to do the least novel work, or at least that the novel work is well bounded and concentrated into one domain.

Fill in the Blank

A theme alluded to above is the idea of filling in the blank - you get the bare minimum working, and flesh out the in-between later. This is not always appropriate, but when it is, it allows LLMs to serve as a powerful tool to get some momentum on a goal. The “blank” need not be a full project, it may be some increment - so as usual, it does well to at least think through what is required to solve a particular problem.

Take, for example, camera support for the Moto G Play in Part 1. I had libcamera, stock camera calibration files, and a working sensor. The “blank” I had to fill was converting these stock calibration files into something that libcamera could at least begin to use for better camera support. The goal (in keeping the “main thing” the main thing) was a working camera that at least looked halfway decent. The useful increment to get there was to make a tool to read stock calibration files and convert them into something that libcamera could read, and make libcamera modifications to support them. That way I could go from stock calibration to a working camera without redistributing stock files.

This camera case is a good low-stakes example - getting it wrong is both immediately evident and harmless: it’s simply a “bad” picture, no harm done. Thus, doing it “right” isn’t terribly important - if it looks good, it is by nature an acceptable result. It also lent itself well to an iterative approach, wherein I used LLMs to trawl GitHub for relevant source code (i.e., the vendor-distributed camera stack from Qualcomm), read calibration files, approximate that same feature (if new) in libcamera and then test on a live device. The tight development loop meant I could use LLMs to work against both captured RAW images and live camera feed and start to zero in on an acceptable result. For more subjective/temporal failures that required human review (unstable brightness, focus breathing) it was still easy to use LLMs to try things, with occasional nudges in the right direction.

I Am Speed

Once you’ve done the “make it work”, you should do the “make it right”, but if it’s low stakes and you’ve instead done the “make it right (enough)”, then why not go ahead and do the “make it fast”? LLMs, when combined with good old fashioned profiling, are great for testing things out.

The key here is to scope the problem: find a user experience that is slow, make sure it can be profiled (in itself something that LLMs could help set up), then you can dig in. It’s a low stakes way to try things, and see some of what’s possible for a given problem. For example, take this PR to flacoutcpp, a tool that searches for the smallest FLAC encoding of a file. Profiling put 93% of its exhaustive mode’s runtime in two loops, and reworking just those made it about 3x faster with byte-identical output. It also added a search level about 90x faster than exhaustive mode, for about a quarter of a percent in file size. Lossless audio kept the stakes low: every change could be checked by decoding and comparing against the source.

Hammer Meets Nail

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

— Abraham Maslow, The Psychology of Science (1966) (Wikipedia)

LLMs are a hammer; learn to tell what is actually a nail. This, depending on experience and personality, will likely be difficult (it is for me). As LLMs get better, it will become increasingly difficult.

I have no solution for this, and I am not a good example of doing this anywhere close to right. For now, I’m seeing a lot of nails.

Treasured Measurements

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.

— Lord Kelvin, “Electrical Units of Measurement” (1883) (Wikiquote)

Put another way:

You can’t treasure what you can’t measure.

You must be able to measure several things for a successful project, some of which include:

Features
Performance
Correctness

Nail these down as soon as you can, and you’ll have more success. Acceptance criteria exist for a reason. At the same time, if you learn something along the way and the goalposts must shift, so be it - but be discerning when doing so; LLMs often make it feel like I have more bandwidth than I really do.

Still a mixed bag, as in Part 1, but here we are anyway.

Part 3 someday, likely once I get a bench setup for a spare EV6 head unit (working on getting 64bit boot).

Notes on LLMs - Part 1

William Floyd — Sat, 17 Oct 2026 00:00:00 +0000

I had written a draft of an anti-LLM post some months ago, and ended up not posting it. Since then, I’ve used these tools much more during my day job, as well as hobby projects. I’ve had many thoughts about LLMs, and their impact on my mental health and ability to learn (or the stunting of said growth). I’ve also experienced many cases where it is undoubtedly an amazing tool that has enabled me to learn new things by reducing the barrier to entry for new fields.

In all, it’s been a mixed bag, and I wish LLMs never existed.

But the technology is here now, so I thought I’d show some of the things I have managed to do aided by these tools. In line with not letting perfect get in the way of good/done, I’ll probably post updates in Part 2 and so on.

Linux Hacking

The Linux kernel is huge, complicated, amazing, and not something I have experience in modifying. I am however quite comfortable compiling software and figuring out how the pieces fit together (eg here and here). That being the case, I’ve gone ahead and tried a couple projects with kernel related hacking.

Samsung Galaxy S9 - `starqltechn`

My old phone collecting dust, so why not try to get it running Nura?¹ There’s already a working port, but lots of peripherals don’t yet work, so I figured I’d start there.

Long story short, using LLMs I was able to get:

Working camera+flash
Aperture control
Manual focus
WiFi+BT (tracked down an existing patch)
DisplayPort output

My Code

Kernel tree
aports (spread across branches)

Vendor Resources

Level 3 service manual (SM-G9600)
Samsung 4.9 kernel source (Droidian’s tree)
Stock firmware dump

Motorola Moto G Play (2024) - `fogona`

Fresh on the success of getting progress on an older phone, I decided to find the cheapest phone that a US resident can just go out and potentially install Linux on. At this time, it’s the Motorola Moto G Play (2024), which can be had for $100 USD new in box, carrier unlocked and bootloader unlockable. There’s already SoC support, but it’s somewhat early.

After much back and forth, I was able to get:

GPU support
Video decoder acceleration
Cameras with PDAF and stock tunings
WiFi, BT
Modem (data, calls, texts/mms)
GPS, accelerometer, compass, rotation, lux, proximity

My Code

Vendor Resources

Level 3 service manual (XT2413)
Kernel: Older, more complete, Latest
Device tree: Older, Latest
Subsystem device trees
- display: Older, Latest
- camera: Older
- audio: Older, Latest
- nfc: Older, Latest
- bt: Older, Latest
- graphics: Older
- video: Older
- wlan: Older, Latest
Drivers
- Motorola modules: Older
- Camera: Older
- Graphics (KGSL): Older, Latest
- Video: Older

PRs that resulted:

Projects that resulted:

bootmanager

Such an inventive name, I know… A Slint GUI + Go backend tailored for Linuxboot+u-root setups, meant to be a beautiful alternative to uboot. Supports touchscreen, SSH, fastboot, LUKS decryption, usage in place of unl0kr, and a web GUI built in WASM from same Slint.

▶ Watch the video

godot-maps

Comaps doesn’t look great on Linux, and I couldn’t find anything else that did what I wanted, so I made a quick and dirty navigation app built on Comaps backend using Godot. It compiles to many platforms too, including MacOS and Android.

go-msm-firmware-loader

Go port of msm-firmware-loader, used by bootmanager

go-qbootctl

A static Go replacement for the A/B slot subset of qbootctl so it works in an initramfs, used by bootmanager

go-chromatix2libcamera

Translates Qualcomm Chromatix tuning files to something libcamera (or a fork thereof) can interpret.

go-unbrick

Swiss army knife that I use to inspect stock firmware images, any files contained therein, and to make some files to help with bricked devices.

go-qshrink4

A reader for Qualcomm QShrink4 hash databases (.qdb / .qsr4), mapping the hashes in QShrink’d modem diag messages back to their format string, file and line.

2024 Kia EV6 Head Unit

New (to me) car, new project! Once I got the stock system image (download from Kia, community knows how to decrypt), a root shell on the running unit (others already figured this out), and the vendor source, I was off to the races.

Given the ancient (3.18.24, from 2015) vendor kernel (for Android Kitkat!), I used LLMs to do minimal mainline porting for the quad Cortex-A53 Telechips TCC803x SoC, which turned out to almost work first try. With a bit of back and forth (and much USB swapping), display out was achieved and I could feed log messages back into the LLM and iterate on fixes. Note that stock was never reflashed, this is a kexec cycle that uses USB data only. The old Android version is in some ways a blessing in disguise, because older systems are in some ways much easier to bend to your will.

I figured the quickest path to something useful was to get bootmanager running, which turned out to be trivial.

Still, after ~3.5min, it would reboot. So using LLMs I logged serial transfers from Android and did some decompilation of the MICOM firmware (seems to be some kind of PowerPC VLE chip) and Android apps to see what signals I needed to respond to, and found a hung SoC check that I can respond to to keep from being reset. At the same time I found signals for things like power draw of climate, battery state, etc. (I still need to map this out properly).

Working:

Display
I2C
USB
eMMC
WiFi (and soon, BT)
Touch
Serial
MICOM keepalive + Vehicle data
re-kexecing back to stock Android

Does it run DOOM?

If you saw the cover image, you’ll know. Of course it does!

▶ Watch the video

Vendor Resources

Mobis open source: “AVNT STD G50 WIDE R6” for “EV6 (CV1)” (AVNT_STD_G50_WIDE_R6.zip, 565 MB)

My Code

Kernel tree

Formerly PostmarketOS ↩︎