G64 images carrying MFM tracks are not loadable by the reference implementation

While reviewing the track-bounds work in #840 I noticed that the firmware stores MFM tracks inside a G64, marked by bit 15 of the per-track length word. As things stand such an image is not a valid G64, and I would like to ask what the plan for those files is.

What the firmware does. disk_image.cc:735 reads the marker as w & 0x8000 and masks the length with 0x3FFF; disk_image.cc:859 and :937 set the bit again when a track is written back.

What the format says. The G64 specification, as carried in VICE's doc/vice.texi, defines that field as one thing only:

$02AC-02AD    Actual size of stored track (7692 or $1E0C, in LO/HI format)

No flag bits, no reserved bits — the only "Reserved" wording in the section belongs to the speed-zone entry. The word MFM does not occur anywhere in the G64 chapter. The format was defined in 1998 by Per Håkan Sundell (CCS64), Andreas Boose (VICE) and Joe Forster/STA (Star Commander), so it is a shared format rather than any one project's to extend on its own.

What the reference implementation does with it. VICE reads all sixteen bits, unmasked, in src/diskimage/fsimage-gcr.c:152:

track_len = util_le_buf_to_word(buf);

if ((track_len < 1) || (track_len > max_track_length)) {
    log_error(fsimage_gcr_log,
              "Track field length %u is not supported.", track_len);
    return -1;
}

With bit 15 set, track_len is at least 32768 against a max_track_length of $1EF8, so the track is rejected and the load fails.

G64 images are exchanged between VICE and the Ultimate and interoperability is expected of them, so this is not theoretical. My position is that such files should not exist, and the reason is the specification itself: the field is defined as a size, the signature reads GCR-1541, and a file making that claim while carrying MFM content does not keep it.

That said, the gap being filled here is real. I checked before writing this, and there is genuinely nowhere else to put this data:

G64 and G71 are GCR by name — GCR-1541 maps to DISK_IMAGE_TYPE_G64, GCR-1571 to DISK_IMAGE_TYPE_G71, and c1541 describes the latter as "VC1571, but in GCR coding".
P64 is a true flux container (NRZI pulses per half-track, range-coded, CRC32), but its magic is P64-1541.
VICE has no low-level MFM path at all: src/drive/iec/fdd.c synthesises the MFM bitstream at runtime from disk_image_read_sector(), writing 0x4e gaps, address marks and CRCs itself. A stored MFM or flux stream is never read.
So MFM exists only at sector level, in D71 and D81, which cannot express weak bits, non-standard sector sizes, deliberately bad CRCs, or anything past track 80 — exactly what copy protection is made of.

And the per-track flag is the right idea. A 1571 has both the GCR path and a WD1770 — VICE's own wd1770.c is titled "WD1770/1772 emulation for the 1571 and 1581 disk drives" — so the drive switches encodings, and a container for 1571 media must be able to state per track which encoding is present. That is precisely what bit 15 expresses. The problem is not the concept; it is that the statement sits inside a file whose signature says GCR and whose length field is specified as a size.

Proposal: keep the idea, move the wrapper.

.h71, magic HYB-1571 — a hybrid-encoding 1571 container. Track-pointer and speed-zone layout as in G64/G71 if convenient, but with an explicit per-track encoding field of its own instead of a bit taken from the length. The MFM payload already produced — sector count, version byte, five bytes per sector, then the data — moves across unchanged, and GCR tracks stay exactly as they are. H rather than M because the file is not MFM; it is both.
.m81, magic MFM-1581 — a pure MFM container. The 1581 has only the WD1772, so no per-track encoding field is needed.

The cost is in the header and the suffix, not in the track data, and a reader that does not know these formats declines them by signature rather than misreading a length as 32768.

One alternative, with a condition attached. The conceptually cleaner route would be to widen P64 — additional magics P64-1571 and P64-1581, suffixes .p71 and .p81. Flux is encoding-agnostic, so the hybrid case disappears, and the format is already documented and would only be extended, never redefined. The condition is that the device must actually have flux or bitstream data. For GCR tracks it does, and GCR bits map to transitions almost directly. For MFM it holds sector lists, so a P64 written from them would have to invent the gaps, address marks and CRCs — producing a file that claims pulse-level fidelity it does not have, which is the same category of problem this issue is about. So: is capturing MFM at bitstream or flux level something the hardware could do? If yes, P64 is the better answer and the rest of this proposal is unnecessary.

If a new format is more than you want to carry, the same specification shows a gentler precedent: G64s from the Kryoflux DTC tool append their extra per-track information as a distinct block tagged EXT ($45,$58,$54), which an unaware reader simply skips. The principle either way is to add a record rather than redefine a required field.

My question, before anyone proposes anything further: do such files exist yet? From the code the population looks narrow — the bit is not produced by conversion, only by mfm_update_callback() setting track_is_mfm when a track is genuinely written back as MFM, after which write_track() persists it. An image from VICE or from a converter never carries it. If none have been shipped or seen in the wild, dropping the extension costs nothing; if they are out there, a replacement needs a migration story.

Lastly, on process. If a new format is the way forward, I would like to bring the VICE side in from the start rather than afterwards — a format only one implementation reads does not solve interoperability, it relocates it. G64 itself was settled that way in 1998 between three implementers. I am happy to open that conversation on vice-emu-mail with you in the thread.

Who knows what users might already have created?

For example, the software Big Blue Reader can format MFM disks on a C64/C128, including PC-compatible formats, and this can be done in emulation as well. So there are perfectly ordinary ways to end up with real MFM disk images.

A new format that addresses the limitations of G64 would certainly be nice. But it would only be useful if the major emulators implement it as well. Otherwise we would essentially end up with the same situation as with an extended G64: a technically good format, but with limited interoperability.

Some of the issues I see with G64 are:

The track length is expressed in bytes, so the exact bit length of a track cannot be represented. This matters for formats where the bitstream does not end on a byte boundary; flux captures can have arbitrary bit lengths.
There is no official representation for MFM tracks.
The format is tied rather closely to the 1541/GCR world and does not provide a good way to represent higher-density formats. Ideally, a new format should not have to invent an entirely different mechanism for every other floppy type.
Multi-speed track representation may also be worth considering. In theory this is useful, although representing and emulating it cleanly can become rather complicated.

So I think the interesting question is not just "can we invent a better format?", but whether there is a format design that is general enough to cover these cases and is realistic enough that VICE and the other major emulators would actually implement it.

Well, Denise, yet another C64 emulator, already supports P71 and, IIRC, P81 as well — so formats for representing non-1541 disk geometries already exist.

That doesn't necessarily mean they are the right solution, but it does mean that, when using those formats, the header has to be compatible as well.

@markusC64

@markusC64, thank you — you were right on both counts, and I was wrong on a point that matters. Let me correct it before anything else.

The bit 15 marker is not a unilateral deviation. I went and looked at Denise after your note, and it reads the same convention. In emulation/libc64/disk/structure/gxx.cpp:

bool mfm = (trackLength & 0x8000) != 0;
trackLength &= 0x7fff;
...
if (mfm) {
    parseMfm(ptr, offset + 2);
}

So at least two independent implementations already agree on it. I opened this issue arguing that such files "should not exist" and that the firmware "should stop doing this"; I withdraw both. Removing it would not restore compatibility, it would break compatibility that is already there. That was a wrong reading on my part, and I am sorry for the framing.

What does still stand is narrower, and I think it is the real problem. The published specification in doc/vice.texi defines that field as "Actual size of stored track ... in LO/HI format" and nothing else, and VICE reads all sixteen bits unmasked in src/diskimage/fsimage-gcr.c:152, rejecting anything above max_track_length. So the files do not load there. The issue is not that someone broke a rule — it is that a working extension exists in two implementations, was never written down anywhere, and a third implementation therefore cannot know about it.

There is already a small symptom of exactly that: the firmware masks the length with 0x3FFF, Denise masks with 0x7fff. Bit 14 means different things to the two. With real track lengths capped at $1EF8 this can never bite today, but it is the kind of quiet divergence that an unwritten convention produces over time.

And my proposal was partly reinventing what exists. Denise already carries .g81 with the magic MFM-1581, .p71 accepting P64-1571 and P71-1571, and .p81 with P64-1581 — I suggested inventing MFM-1581 without knowing it was already implemented. Your remark that "the header has to be compatible as well" makes complete sense to me now.

So the question I should have asked is the one you actually posed: not "can we invent a better format", but whether there is a design general enough to cover bit-exact track lengths, MFM, higher densities and multi-speed tracks, and realistic enough that the major emulators would implement it.

What I would like to suggest, with all due humility: that the people who actually own this problem talk to each other — you, @GideonZ, piciji as the author of Denise, and the VICE side. In the interest of full transparency: I sent a mail to vice-emu-mail@lists.sourceforge.net earlier today asking whether there is prior art here and whether there would be any appetite for a common format. That was before I knew about P71, P81 and G81, so the question was less well informed than it should have been. If it is better for that conversation to be led by people with more standing in this area than me, I would genuinely prefer that.

A personal word, since I am the intruder here. I have been in this codebase for a few weeks and you have all been at this for decades; g64conv is cited in the firmware's own comments, and the error byte in the MFM sector header is there because you asked for it. I am grateful for the work all of you have put into this scene long before I turned up, and for the patience shown to me in these threads. I know of no animosities between anyone involved, and I hope there are none. If there are, or if my walking into this topic is stepping on something I cannot see, please say so plainly and I will withdraw from it — the format question matters far more than my part in it.

Well, for the multi speed tracks I am not sure if an emulator could handle them in real time. The problem I see is moving the head to a neighbour track and knowing the position in the data. On the other hand, as storage for analyzing them might be a good point... The multi spped track feature is very debatable.
My POV is: The format should offer it for its use cases, but you aren't required to use the speed information in case there is no idea how to do it in real time. IMHO better than a new format that has restrictions that forces for yet another extension or yet another new format. But again: YMMV.

@markusC64

@markusC64 Fully agreed. Any new format intended for broader use should be developed in collaboration with other implementations. Otherwise, even a technically sound design risks further fragmenting the ecosystem and consuming development effort without delivering interoperability.

More generally, the project now has around 300 open issues, while only about five firmware developers have raised PRs in the past three months. This makes it increasingly difficult to keep pace. This issue and the discussion are entirely valid, but I think we would also benefit from collectively spending more time triaging the backlog and addressing the highest-priority issues.

Improving regression coverage for the existing feature set would be particularly valuable. PRs that add focused tests for current behaviour would help prevent regressions and enable us to deliver future changes with greater confidence and efficiency.

@GideonZ

Still on this; the first step turned into three parts, and some of it is in #840.

1a — test material, our debt and not VICE's. groepaz on the mailing list: "I already asked people to provide test programs for those things back when MarkusC64 invented those hacks for 1541U — but so far no one was interested in making them." One correction in fairness to both names: the commits that put MFM tracks into a G64 are @GideonZ's, 10 June 2021 (2c17e810). @markusC64 shaped the design — the error byte is there because he asked for it — but whose convention it is was never written down either. #840 now builds mixed GCR/MFM G71 images in code and mounts them on hardware; next is to let the firmware write an MFM track itself, Big Blue Reader on a 1571, and check it matches, so we are not just agreeing with ourselves. @markusC64, if you have images or tooling that would widen that, I would rather build on yours; g64conv reaches flux, which we cannot.

A question that decides what bit 15 costs: does anyone know a G64 with a track long enough to need bit 15 as length? That is 32768 bytes, roughly 4.3 revolutions of the fastest zone. Across 1302 images here — 55,104 length words — the longest track is 7928, every header declares 7928, and no word sets bit 14 or 15 for length. One counter-example changes the answer, which is why I am asking rather than concluding.

1b — a decision for you two, before anyone writes code. @GideonZ, @markusC64: keep bit 15, or move new writers to an EXT-style block? G64 already carries that extension point — tag EXT, a version byte, sixteen bytes per track, with a format code at $0e and a bitcell size in nanoseconds at $08. A reader that does not know it never reaches it, because track data is found through the offset table, so an unaware VICE would load such a disk and read its GCR tracks instead of refusing the file. Bit 15 cannot do that and costs half the length range on top. Against it: nobody implements EXT today, its format codes are SPS's registry of protection schemes rather than encodings, and the payload would still look like a track with no sync to an unaware reader.

This is a niche inside a niche — .d64 dwarfs .g64, and the marker is only written when an Ultimate writes a track back as MFM — so the cheapest path that works is the right one, and I have no view on which that is. If you want to keep bit 15, writing it down is cheap and I will do the writing.

1c — the gap in VICE, once 1b is settled. "the 1571 emulation doesn't even implement MFM yet — so someone must implement that first" is fair. Correct me if I have misread it: the registers look mapped already (memiec.c, $2000-$3000 → wd1770d_read/store), wd1770.c is titled for the 1571 and the 1581 alike, and fdd.c already synthesises MFM with sync marks, gaps and CRC. Missing is the wiring — wd1770_reset() runs only for DRIVE_TYPE_1581, wd1770_attach_image() takes only D81 and D1M, and the generator reads per-drive constants where a G64 track needs per-track ones. If that is the shape of it, I will do the work rather than ask for it — but which shape it takes depends on 1b, which is why I am not starting it yet. Tracks > 35 is a separate and larger question and I am not claiming an answer for it.

Meanwhile #840 fixed the part that was plainly broken on our side: the length was read from fourteen bits and written with fifteen.

@markusC64

@markusC64 — a direct question, since the mention in my longer comment above was easy to miss.

We found a way to make a real 1571 write MFM tracks from a plain C64, without a C128: the drive's own burst command FORMAT MFM. The 1571 ROM documents it as conventional protocol with no burst transfer (fastutl.src, "BURST CMD FOUR - FORMAT MFM"), so a few lines of BASIC over the slow bus are enough:

OPEN15,8,15,"U0>M1"
PRINT#15,"U0"+CHR$(102)+CHR$(129)+CHR$(1)+CHR$(2)+CHR$(39)+CHR$(9)+CHR$(0)+CHR$(0)+CHR$(229)

That is the PC 360K layout: both sides, index mark, 40 tracks of nine 512-byte sectors starting at 1, fill $E5. Every track goes through WD177x Write Track, so on a C64U it exercises exactly #921/#937. I will run it there with a G71 mounted and look at what the firmware stores.

What I cannot produce myself is the reference to compare against, and that is where your tooling reaches further than ours:

A G71 of the same format from a real 1571, ideally captured from flux with g64conv, so the comparison is against hardware rather than against our own generator.
Your view of the stored layout: whether an MFM track as the firmware writes it (bit 15 on the length word, sector count, then 5-byte sector entries) is what you had in mind when you shaped it, so that we check against the intent and not only against the code.

For sector writes there is also RUN's "MS-DOS/C-64 Connection" (Garamszeghy, 1990), which writes single MFM sectors through the 1571's job queue ($A4/$A6). It is useful later, but it does not format.

@markusC64

Following up on my note from 01.10 ("I will run it there with a G71 mounted and look at what the firmware stores"), now done on an Ultimate II+L in a C64, official 3.15a (dddd29b2), drive B as a 1571.

A blank G71 (80 slots of 7928 bytes, cylinders 0–39 on both sides), then the 1571's own burst FORMAT MFM from the C64: U0>M1, then U0 + $66,$81,IL,2,39,9,0,0,$E5 (PC 360K: both sides, 40 cylinders, nine 512-byte sectors from 1, fill $E5). Answer 00, OK,00,00 after 23 s, for interleave 1 and for 0. All 80 tracks are stored the same way:

length word 0x9EF8: bit 15 plus 7928, the slot's full size, not the 4770 bytes the content needs (2 + 32×5 header + 9×512). The rest of the slot keeps its old bytes. So for an MFM track, the 15 bits give the reserved space, and the content follows from the sector count.
sector count 9, version byte 0; entries (cylinder, side, sector, size code 2, error 0) with the right cylinder and side on every track; data all $E5.
the table is in physical order, as the drive wrote it: interleave 1 gives 1,6,2,7,3,8,4,9,5, interleave 0 gives 1…9. That is the 1571's own definition (sectcv in MFMCNTRL.SRC steps by interleave + 1), so the firmware keeps what the WD177x was given.

For a reference from real hardware: fluxfox's test set has a real PC 360K disk with the same geometry (40×2×9×512, IDs 1–9 in physical order, 250 kbit/s MFM) as flux (SCP, KryoFlux stream), bitstream (HFE, HxC MFM) and IMD: https://github.com/dbalsom/fluxfox/tree/main/tests/images/sector_test. With interleave 0, the G71 above matches it in cylinder, side, sector IDs, size and order; only the fill differs. It is from a PC drive, not a 1571, so it does not replace a G71 from a real 1571. But it gives the low-level target that a G71 MFM track has to reproduce.

@markusC64, @GideonZ: the stored layout above is what the firmware does today. Is the slot size in the length word what you intended, or should the 15 bits hold the content length? That is one of the points the spec text has to settle.

Navigation Menu

Repository navigation

G64 images carrying MFM tracks are not loadable by the reference implementation #920

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

G64 images carrying MFM tracks are not loadable by the reference implementation #920

Description

Activity

markusC64 commented on Sep 18, 2026

markusC64 commented on Sep 18, 2026

enver-haase commented on Sep 18, 2026

markusC64 commented on Sep 18, 2026

chrisgleissner commented on Sep 19, 2026

enver-haase commented on Sep 25, 2026

enver-haase commented on Oct 1, 2026

enver-haase commented on Oct 7, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions