Smartport hard disk emulation is now working on Floppy Emu, with my latest firmware! This mounts an SD memory card as an external 32 MB hard disk, on Apple II machines with Smartport support: the GS, or //c with ROM version 0 or later. It provides solid state storage similar to the CFFA 3000, Focus Drive, etc. I’ve tested it on both the GS and the //c, and it works without any special adapters, using the connector and cable I’ve already been selling for Macintosh use. While there are still many wrinkles to iron out, the basic functionality is there, so I’m excited.

I’m not releasing the new firmware just yet, because it’s still so full of holes that it’s probably not useful to anyone. Here’s where things stand at the moment:

• Emulates a single hard disk, using a disk image stored on the SD card named “smart.hdv”. This is just a plain ProDOS-ordered disk image file. While Smartport technically supports disks up to 8 GB, I believe ProDOS and GS/OS are limited to 32 MB maximum.
• Disk images are mounted read-only. Write emulation coming soon!
• The disk won’t “cold boot” when you turn on the power. Instead, you have to turn on the Apple II, wait a moment, then press CTRL+Open Apple+RESET. This is because the emulator doesn’t initialize itself fast enough at first power-up in order to be ready when Apple II first checks the disk. I hope to have a fix for this soon.
• Under GS/OS, when using Floppy Emu as a Smartport hard disk, a phantom 5.25 inch drive also appears on the desktop. I’m not sure what’s happening there…

The current version of the firmware spews a lot of text to the LCD to help troubleshoot what’s happening. R for a block read, S for a status request, I for an init command, etc. If necessary, it can also display the block number for the read commands, and other details. But it’s a lot of information to cram onto a 21 x 6 character LCD, especially when it all scrolls by so fast you can barely read it.

 
ProDOS Hard Disk Images

Beyond those bugs, there are a couple of larger issues that also need sorting out. For the Apple //c, I’m actually not sure how to make a 32 MB hard disk image with ProDOS and a collection of other software. The //c can’t run GS/OS, so I can’t use any of the many GS hard disk images available on the web. There must be some way to use Ciderpress and/or an Apple II emulator to create a blank 32 MB disk image, format it, and fill it with ProDOS and other software, but I can’t seem to figure out how.

The photo shows an Apple //c booted with Floppy Emu’s new Smartport firmware, using a ProDOS disk image that I downloaded from http://www.apple-2.com/vintage-dl/smartporthd.zip. The description of the disk image says it’s 32 MB, but the image file is actually only about 6 MB uncompressed. What’s even stranger, you can see in the photo that it reports 57422 blocks used and 20466 blocks available, which is a total of about 38 MB – larger than either the actual or claimed size of the disk. I believe this info is taken from the disk image’s filesystem data and not from the drive itself, so I’d like to try making another disk image for the //c and see if the size is reported correctly.

 
What’s the Status?

The other issue is the Smartport STATUS command. There are a few different commands that the computer can send to a Smartport device, including init (command 5), read (1), and status (0). In my tests, the computer requests the drive status a crazy number of times, sometimes making 14 consecutive status requests. Since the data returned is just some capabilities flags and the drive’s size, there’s no reason to request it more than once. I strongly suspect that my status reply is somehow not right, or not what was expected, so the computer re-requests status over and over hoping for a different result. Eventually it makes an extended status request (command 0x40), even though I’ve set the drive capability flags to indicate it does not support extended commands. I respond to this with a generic NAK that I’m not sure is a valid response, but it seems to work and the computer eventually boots after making tons of status requests.

I’m not sure how to go about debugging this, or whether it’s even abnormal. I can troubleshoot low-level issues, like checksum errors or data being read incorrectly. But when the problem is at a higher level in the OS software or Smartport ROM routines, it becomes a black box that I can’t see inside. What kind of status response was the software requesting, and why wasn’t my response satisfactory? Or was it? I probably need to find some kind of Smartport utility or diagnostic software, if such a thing exists, to help troubleshoot this one further.

 
But Wait!

Update: it appears the disk image size is being reported correctly, and the status is probably OK too. I was confused because the disk image I tested on the //c does something strange. In its ProDOS header it says it’s 32767 blocks (just under 16 MB), but there’s only 6 MB of data in the image, and the image file itself is only 6 MB. The 10 MB of empty zero space that should be in the image file is just missing. This is fine as long as you treat the image file as a read-only disk, but if the computer ever tried to write new files into the 10 MB of empty space, it would blow up.

Through experimentation, I discovered that Apple II System Utilities (shown in the photo) reports the number of free blocks correctly, but it doesn’t actually know how many used blocks there are. Instead, it takes the total number of blocks on the drive (as reported by the Smartport status command), and subtracts the number of free blocks. In this case, I was using the size of the disk image file to determine the size of the disk drive to report for the status command. So it looked like there was a 6 MB disk drive with about 10 MB of free space, meaning there was -4 MB available! But the negative number was displayed as a positive unsigned count of blocks used, leading to confusion.

Along the way, I found that changing the disk drive size in the status reply caused the expected change in the “blocks used” count of Apple II System Utilities. That means the status reply was actually being received OK by the Apple II. I also found that changing the read-only flag in the status data worked as expected. And I could modify other status values to identify the drive as a removable drive or a fixed drive. When identifying as a removable, the Apple II continued to poll status about once per second after booting up, presumably to check if the media had been removed. But when identifying as a non-removable drive, the Apple II made no more status inquiries after booting up. All of this tells me that the status info is reaching the computer just fine, and is being interpreted as intended. I still don’t know why the computer requests the status so many times during the boot process, but maybe it’s normal. At any rate, it doesn’t cause any problems that I can see.

Some progress at last: Floppy Emu is now partly working as an external Smartport hard drive for the Apple II GS. This should provide functionality similar to the Focus Drive, CFFA 3000, or other solid-state hard drive solutions, for Apple II Smartport-capable machines: the II GS, and the IIc with ROM versions 0 or later. There’s still some weirdness going on, and the new firmware doesn’t yet work on the IIc, and only half-works on one of my two GS machines, but there’s been enough progress that I’m confident I’ll be able to get the rest of it sorted out. It boots my plain vanilla GS machine with no other drives attached, using a 32 MB hard drive image of GS/OS 6.0.1.

Sorry this is a mostly content-free update. I’m still alive and working on this, and that’s about all you need to know for now. I hope to post more details soon once I have something working that’s a bit more robust.

Judging from the Apple II questions I’ve received, my commentary has left a lot of people confused. Sorry everyone! Here’s a recap of Apple II disk support for Floppy Emu. I’ll try to clarify the current status of Floppy Emu disk emulation, and the plans for the future.

As of today, there are two separate firmwares available. One supports the Macintosh and Lisa (floppy and hard disk), and the other supports the Apple II. Both will work on any Floppy Emu board, and can be downloaded from the product description page. Installing new firmware only takes a few minutes – just copy two files to the SD card, and press a few buttons. You can switch between the two firmwares as often as you’d like.

The Apple II firmware will eventually emulate three kinds of disks:

• 5 1/4 inch (140K) disks, for any Apple II machine
• 3 1/2 inch (800K) disks, primarily for the Apple IIGS
• Smartport hard disks (32MB), primarily for the Apple IIGS and some models of Apple IIc

5 1/4 and 3 1/2 inch disk emulation is finished and working in the current Apple II firmware. Download it now and go crazy! Work on Smartport hard disk emulation hasn’t started yet, but it’s next on my “to do” list.

 
Hardware Connections

The Floppy Emu board can be connected to an Apple II system in two different ways, depending on your needs. Most people will use the 19-pin disk port. The Apple IIGS and IIc have built-in disk controller circuitry, with a 19-pin disk port on the back of the machine. The Apple 5.25 Drive controller card has the same 19-pin disk port, and will fit in any model of Apple II with slots. The Floppy Emu board can connect to this 19-pin disk port. If you have the Emu with built-in floppy connector, then the board will plug directly into the disk port, and hang off the back of the machine. Or if you have the Emu with convertible extension cable, the adapter at the end of the cable will plug into the 19-pin disk port.

If you have an older Apple II with the Disk II controller card instead of a 19-pin disk port, you can also connect the Emu to one of the 20-pin headers on the Disk II controller card. Use the ribbon cable that came with the Emu convertible extension cable, or borrow a ribbon cable from a Disk II drive. All Emu boards have a 20-pin shrouded header where the cable will plug in. Be careful to orient the other end of the cable correctly, since the controller card’s 20-pin header isn’t keyed or shrouded, so it’s easy to accidentally connect the cable offset or backwards. The red stripe on the cable should go to the pin marked “1” on the 20-pin header.

 
Hardware Mods

During development of the firmware for 5 1/4 inch and 3 1/2 inch Apple II disk emulation, I discovered that some Apple II setups can’t be supported through firmware changes alone. In addition to new firmware, these setups also require some external hardware modifications for the Floppy Emu board. The easiest way to handle this is with an Apple II adapter board that connects inline between the Floppy Emu and the Apple II computer. Something that looks roughly like this:

I plan to offer an Apple II adapter board as a new product, probably beginning this summer. I’m intentionally going slowly, postponing manufacturing of the adapter boards until the Smartport hard disk work is finished, and I’ve done enough testing to be confident there are no other issues that require hardware modifications. Once it’s ready, the Apple II adapter will be available as a separate item, or bundled with an Emu board for new customers.

As pictured above, the Apple II adapter board is a 19-pin to 20-pin adapter. This will be fine for most Apple II customers, but will be useless for customers using the Disk II controller card, since it doesn’t have a 19-pin disk port. I’m undecided whether to add a second 20-pin header to the existing adapter board design (increasing the size and cost for everyone), or make a different version of the adapter board (more product fragmentation), or something else.

Not every Apple II user needs an adapter board – only the “some setups” that I mentioned. But for those who want a simple and fool-proof path to Apple II disk emulation, the adapter board will be the way to go. For power users willing to read the fine print, they may find that an adapter board isn’t necessary for them. The current landscape looks like this:

5 1/4 disk emulation – Reading from a disk image works without the adapter. Writing to the disk image also works without the adapter on the IIGS and IIc with built-in disk controller. The adapter is required when writing to the disk image using a separate disk controller card, like the Disk 5.25 or Disk II controller cards.

3 1/2 disk emulation – Reading and writing to a disk image works without the adapter, on the IIGS ROM version 1 (the most common version), when the computer is booted from the emulated disk. The adapter is required for the IIGS ROM version 3, or for any Apple II model when the computer is booted from a different floppy or hard disk.

 
DIY

Rather than wait for an adapter board, some people may want to make hardware modifications themselves. There are two separate modifications: one for 5 1/4 disk emulation and one for 3 1/2. The 5 1/4 inch modification is relatively easy for anyone who’s comfortable with a soldering iron, but the 3 1/2 modification is a bigger challenge.

The 5 1/4 inch modification requires only a single 10K ohm pull-up resistor, connected between /WRREQ and +5V. This won’t interfere with 3 1/2 disk emulation, or Mac/Lisa disk emulation, so it’s fine to leave permanently installed. You can make this connection anywhere on the board that’s convenient. The easiest spot is on the bottom of the board, near the disk connector. But a resistor added here will add a little bump that prevents the board from fitting properly in the standard case. If that’s a concern, the pull-up resistor can also be added on the top of the board, using 30 AWG patch wire to reach the finely-spaced pins of the CPLD chip. Using these photos as a guide, connect a 10K resistor between any pair of /WRREQ and +5V pads:

The 3 1/2 inch modification requires adding additional chips, inserted inline between the Floppy Emu board and the Apple II. A schematic is shown here. Unlike the 5 1/4 inch modification, this change is only appropriate for emulation of 3 1/2 inch Apple II disks. You’ll probably want to add a switch to disable the additional circuitry when emulating other types of Apple II disks, or Macintosh/Lisa disks.

 
Don’t Forget to RTFM

Remember, many Apple II setups don’t require any hardware modifications at the Emu board – they’re just plug and go. Before you heat up your soldering iron, or cry over the wait for an Apple II adapter board, read the details above and see if you actually need one. If in doubt, try it first and see. You won’t hurt anything, and the worst case is that it simply won’t work. If 5 1/4 inch writes cause the Emu to freeze, or 3 1/2 inch I/O causes the Apple II to freeze while booting or report strange boot-up errors, then you’ll know it didn’t work. Everyone should at least be able to do 5 1/4 inch disk read emulation, which means you can boot up your favorite games like Castle Wolfenstein straight from the SD card. Have fun!

Wah wah wah waaaaaaah. In the battle for 5.25 inch Apple II disk write emulation, it seems I declared victory too soon. While write support works great in many situations, it doesn’t work at all in others. The problem is with the /WRREQ signal from the Apple II, which goes low to indicate when a write is occurring. On old-school disk controller cards, like the Disk 5.25 Controller Card, or the even older Disk II Controller Card, this signal is generated by a 74LS05 hex inverter chip with open collector outputs. It can actively pull the signal low, but it needs an external pull-up resistor to pull the signal high. And Floppy Emu doesn’t have one. On the Apple IIgs (and probably also the IIc), the circuitry can actively drive /WRREQ high, so this isn’t a problem. And on other Apple II models, it’s also not a problem if there’s another real drive connected in the daisy chain, because that drive will have a pull-up, and /WRREQ is shared among all drives. But if Floppy Emu is the only disk drive connected to a Disk 5.25 or Disk II Controller Card, writes won’t work. Ouch.

The logic analyzer trace above shows what happens. /WRREQ is connected directly to an input pin on the Xilinx XC9572XL CPLD. The input has a very weak pull-up that’s active at initialization time, as well as a keeper circuit that actively maintains the input at the last externally-driven level. At power-up, the weak pull-up pulls /WRREQ high, and the keeper circuit keeps it there. Then when a write occurs, the disk controller card actively drives /WRREQ low. At the end of the write operation it releases /WRREQ, but with nothing to pull it high again, the keeper circuit maintains the low value forever. To the Floppy Emu, it looks like a write operation that never ends. You can see the result in the trace. /WRREQ stays low forever, even after the drive enable signal is de-asserted.

Unfortunately, I don’t think there’s any way to fix this other than to add a pull-up resistor. The XC9572XL CPLD chip doesn’t have configurable built-in pull-ups that I can turn on in firmware, as far as I can tell. 5.25 inch read emulation will still work fine, as well as 5.25 write emulation on a IIgs or on any Apple II with another drive present. But for a single-drive Floppy Emu setup on an Apple IIe, write emulation won’t work without a hardware fix.

I already designed an adapter board, which I described in this earlier post. I had hoped that the adapter board would only be needed for certain situations of 3.5 inch Apple II disk emulation, but now it seems certain situations of 5.25 inch emulation will need it too. There’s too much “certain situations” about the whole thing for my comfort, and I’m sure it confuses everyone else just as much. So I’ll probably just rechristen the adapter board as a general purpose “Apple II adapter”, even though it’s not strictly needed for all circumstances of Apple II emulation. That will be easy to understand, and won’t steer anybody wrong. Power users can read the footnotes and discover which Apple II setups can work without the adapter board, if they wish.

Adding a pull-up resistor to the adapter board will be easy, but it does mean I’ll need to have new PCBs made. The only good thing in all of this is that it justifies my decision to postpone manufacturing large numbers of the adapter boards until after the Apple II 5.25 inch and Smartport disk emulation is working. If I had built lots of adapter boards three weeks ago when I thought they were only needed for 3.5 inch Apple II disks, I would now have a lot of angry people who’d bought them, and a pile of additional boards I couldn’t use.

Whew! Floppy Emu write emulation for 5.25 inch Apple II disks is finally working. That was a painful issue to resolve, because writes to the 5.25 inch emulated disk were working fine in my controlled tests, but things went haywire when I tried a simple DOS write operation. Argh! I went a little crazy jotting down notes about possible causes, theories, and doodles – my standard way of organizing my thoughts for problems like these. Not that it seemed to help.

I finally hit upon the idea of starting from a reference DOS disk image, and performing the same file save operation in two different settings: once on a regular Apple II system with Floppy Emu, and once in the AppleWin machine emulator. Then I could use a binary diff tool to compare the resulting modified disk images from each case, and see how they differed.

What I found was that a single byte differed between the two disk images, after performing what should have been an identical file save operation. Oddly, the byte was in a sector that wasn’t even supposed to be modified by the save operation. The diagnostic info I collected from Floppy Emu said that the Apple II never modified that sector, but there it was anyway. In a sector that should have been all zeroes, the second byte was a 1. And the effected sector was immediately after another sector that was modified by the Apple II. Hmm.

You can probably guess what the issue was: buffer overflow. Whenever a sector was written, the first two bytes of the next sector in memory would also get overwritten. This problem was invisible unless the effected sector happened to lie inside a range of two non-contiguous dirty sectors on the same track, in which case the whole range of sectors would get written back to the SD card as a performance optimization. This never happened during my simple single-sector write tests, which is why I could never find a problem there. The issue only occurred during a more complex write operation involving non-contiguous writes on multiple tracks of the disk.

In truth, the buffer overflow was only the last of three separate problems that prevented 5.25 inch writes from working correctly, but it was the largest bug and the others aren’t interesting enough to describe. So now it works – yay! I tested booting from an emulated DOS disk and saving some BASIC programs to it. I also tried booting GS/OS on the IIgs, and using it to delete the contests of an emulated ProDOS 5.25 inch disk, copy a new program file onto it, and run the program.

 
Try It

Like the Mac and Lisa firmware, this new Apple II firmware supports writing to the data portion of each sector, as occurs during standard file I/O operations. Writing new address headers for the sector is not supported, which means it’s not possible to format the emulated disk or do bit-copy writes with tools like Copy II+. If you find yourself wanting to format the disk to get a new blank image to work with, I’ve included a few different blank disk images with the latest firmware.

Download the new Apple-II-0.1D-F3 firmware, and try it for yourself! As with the earlier Apple II firmware versions, use this firmware only when connected to an Apple II computer. If a Floppy Emu board running the Apple II firmware is connected to a Mac or Lisa, it may cause damage.

Please let me know what Apple systems and disk images you’ve tried, especially if you run into problems. Have fun!

The updated Apple II firmware for Floppy Emu is now available! This update adds 5.25 inch 140K disk image support, and is compatible with any Apple II model, using the standard Floppy Emu hardware. Download apple-II-0.1C-F3 and try it out! I also included a few sample disk images, so you can be running Oregon Trail and a few other classics right away. The firmware supports standard Apple II 140K disk images in .PO, .DO, .DSK, .NIB, or .2MG formats, as well as Apple II 800K (3.5 inch) disk images in .2MG or Disk Copy 4.2 formats. Press the SELECT button from the Emu title screen to switch between 5.25 inch and 3.5 inch disk emulation modes.

Some important notes:

• I’m having difficulty with 5.25 inch disk write emulation, so this firmware treats 5.25 disk images as read-only. I hope to add full read/write support for 5.25 inch disks in another firmware release soon.
• The Floppy Emu board can be connected directly to the 19-pin floppy connector on the Apple IIgs or Apple IIc. For the Apple II, II+, and IIe, the Emu board should be connected to your disk controller card: either the Disk 5.25 Controller Card with 19-pin connector, or the classic Disk II Controller Card with two 20-pin connectors. If using the Disk II Controller Card, be careful to orient the cable correctly, since the card’s connector is not keyed and it’s easy to accidentally connect the cable offset or backwards. The red stripe on the cable should go to the pin marked “1” on the Disk II Controller Card.
• This firmware includes support for 5.25 inch and 3.5 inch Apple II disk emulation. 5.25 inch disk emulation works on a stock Emu board, and does not require any adapter. Full emulation of 3.5 inch disks requires an adapter board that I plan to release soon. Without the adapter board, 3.5 inch disk emulation works on a ROM 01 Apple IIgs when booting from the Floppy Emu, but not on a ROM 03 IIgs or when the Emu is not the boot disk.
• As with the earlier Apple II firmware versions, use this firmware only when connected to an Apple II computer. If a Floppy Emu board running the Apple II firmware is connected to a Mac or Lisa, it could cause damage.

 
NIB Images

This firmware introduces support for NIB images, which contain the raw disk bytes stored on a physical floppy disk, instead of the high-level sector data contained in other types of disk images. With NIB images it’s possible to store and emulate many types of copy-protected disks, including disks with non-standard sector headers, custom sector sizes and encoding, and all sorts of other crazy schemes. This is important for the Apple II world, because the use of disk-based copy protection was common among Apple II software. A disk emulator that didn’t support NIB would be limited to only using “cracked” versions of the software in which the copy protection had been removed.

Implementing robust NIB support is challenging. I’m not sure if other Apple II disk emulators also support NIB format, but at least one that I checked does not, so I’m happy that I was able to make it work for Floppy Emu. After digging into the details, I can understand why some designers may have chosen to skip NIB. The NIB format represents a track as 6656 disk bytes, but this is insufficient to represent the full structure of a real floppy disk track. On a real floppy, most disk bytes contain 8 bits, but some disk bytes called sync bytes are actually 10 bits long. Of course a 10-bit quantity isn’t a true byte, but the term “sync byte” is used regardless. The sync bytes are FF (hex) followed by two zero bits, so in binary they look like 1111111100. The problem is that sync bytes are stored in the NIB file as plain 8-bit FF bytes, and the knowledge that a particular FF is a sync byte instead of a standard FF data byte is lost.

I believe the NIB format was developed mainly for use with PC-based Apple II emulators, and not for use with real Apple II hardware. While I’ve never investigated how emulators like AppleWin are designed, I would guess it’s impossible for the emulated disk controller to get out of sync with the emulated disk, so the absence of proper sync bytes in the NIB data is unimportant. But when running on real Apple II hardware, such as with the latest Floppy Emu firmware, the absence of sync bytes is fatal. To solve this problem, I’ve designed the firmware to make an educated guess as to which FF bytes in the NIB are actually sync bytes, and which are just plain data bytes. In practice, this seems to work great most of the time, but copy protection schemes are often designed to intentionally thwart exactly this kind of analysis. In my tests, all the copy-protected NIB images that I tried worked, save for one intermittent problem with Moon Patrol on the Apple IIgs. Yet the Moon Patrol NIB worked fine on my Apple IIe, so maybe it’s some kind of IIgs incompatibility rather than a NIB issue.

 
Write Support

Support for writing to emulated 5.25 inch disks has proven to be much more troublesome than I’d expected. Single-sector writes appear to be working fine: I can use a sector editor tool to view and modify individual sectors on the emulated disk, and the data gets modified correctly. That means there aren’t any low-level problems involving timing or checksums, which is great. Yet when I try a higher-level write operation, such as copying a file, everything blows up. The exact nature of “blows up” is hard to identify – if I could explain it, I could probably understand how to fix it. In practice it means that the software complains about “write error”, or the Emu displays a buffering-related error.

On the Macintosh, all floppy disk write operations are verified by reading back the newly-written data, unless verification is explicitly disabled. From the point of view of a disk emulator designer, that’s a very nice feature, because any mistakes that occur during writing are caught and reported right away. But for Apple II software, verification of writes doesn’t appear to be the norm. At least in the GS/OS Finder, you can duplicate a file on a floppy disk, and GS/OS will show that everything succeeded regardless of what was actually written to the disk. If there was a problem, you’ll only find out about it later when you try to use the new file.

A second wrinkle is that some Apple II copy software like Copy II+ does something more akin to a single-track format operation than a standard write. The sectors on a disk consist of an address header followed by a data chunk. Normally the address headers are written once when the disk is formatted, and then never touched again – only the data chunks are modified. My experiments show that Copy II+ violates this rule, though, and writes new address headers. Floppy Emu isn’t designed to handle this, so the write operation fails. This isn’t an Apple II issue, but is also true of the Mac and Lisa firmware, and has been the case since the earliest days of Floppy Emu. There may be a way to configure Copy II+ so it doesn’t rewrite the address headers, but if so I haven’t found it yet.

I found more unexpected results when snooping on ProDOS disk writes. ProDOS works in units of 512 byte blocks, which are stored as two separate 256 byte sectors on the floppy disk. Every ProDOS write should therefore look like two successive sector writes, to two different physical sectors. But what I observed was that the same physical sector was written twice in a row. This must be a bug somewhere in my layers and layers of abstractions, but I haven’t been able to locate it. Or maybe the way I’m tracking the write activity is flawed.

In short, write support is a pain. I can see when something went wrong by observing error messages, but I only have a limited ability to look inside and examine each step of the write process to find the cause. I may need to invest some time in creating better debugging tools, before I can get to the bottom of it all.

 
Feedback

As always, I welcome your feedback. If you’ve tried this new 5.25 inch disk emulation on your Apple II system, leave me a note in the comments! Even if you have nothing to say beyond “works on my IIc”, that’s still helpful to know. Thanks!