Nobody enjoys troubleshooting non-working hardware, and I’m no exception. Every time I ask a contract manufacturer to assemble a batch of more Floppy Emus, there are always a few that don’t pass QA testing. What happens with those? For several years the accumulating QA failures have been sitting in a pile in the corner of my office, along with a few customer returns, all waiting for the day when I would dedicate time to their investigation. It was a long time coming, but “Failure Analysis Day” finally arrived, or more like Failure Analysis Month, and the results were pretty interesting.

Here’s a breakdown of the unique causes of failure that I identified, and the number of boards affected by each cause.

 
Clock Crystal

This was a strange failure that required a lot of time to track down initially, but once I learned to recognize the symptoms, I realized that most of my QA failures were due to this problem. I wrote about the clock crystal mysteries in more detail in a separate post last month. The short story is that during my most recent manufacturing batch, my normal supplier of clock crystals was out of stock. The contract manufacturer, with my approval, substituted a different crystal with the same specs. It shouldn’t have affected anything. But somehow it did.

26 of the QA failures appeared to have no functioning clock at all. The board utterly failed to do anything when the microcontroller clock source was changed to the external crystal. A further 16 displayed some level of function, but with erratic behavior or failures at higher clock speeds. Initially I wasn’t sure whether this was due to the newer crystals exposing some defect or fragility in the Floppy Emu design, or whether it was simply caused by defective or damaged crystals. Eventually I came to the opinion that the crystals were damaged by rough handling or overheating during assembly. Replacing the crystals and reprogramming the boards resolved all the issues.

 
CPLD Chip

The Xilinx CPLD chip on the Floppy Emu is a delicate flower that has long been a source of challenges. Prior to this year’s crystal-gate debacle, the CPLD was the single biggest source of failures that I’d observed. As the chip that’s directly connected to the Floppy Emu’s external interface, it bares the brunt of any static discharge or electrical stress. It’s a 5V-tolerant 3.3V part, but its 5V tolerance has sometimes seemed a bit questionable, at least in the way it’s used here.

Symptoms of a failed or bad CPLD can include disk emulation failures, overheating, or erratic behavior. Usually the device will still be functional and text appears on its display, but the disk features no longer work. In extreme cases the failed CPLD acts as a hard short-circuit from power to ground, and then nothing works. Replacing and reprogramming the CPLD resolved all of the problems with these boards.

 
Microcontroller not programmed

Amusingly, or depressingly depending on your perspective, the third leading cause of QA failures was that the microcontroller simply wasn’t programmed. Somebody fell asleep at the switch at the contract manufacturer, lost track of what they were doing, put a PCB in the wrong pile, or whatever. A board with an unprogrammed microcontroller will appear completely dead at first glance, but it still responds in the debugger and it only takes a few seconds to flash the chip and get everything working.

 
Soldering problems

Every component must be electrically bonded to the PCB with solder. Soldering problems can be tough to spot with the naked eye, but usually jump out under magnification, so one of my first troubleshooting steps is usually to look at a problematic board at 10x. I really should get a cool desktop microscope, but for the moment I’m using a cheap 10x jeweler’s loupe which works well enough.

A couple of boards had too much solder in places, resulting in a solder bridge that unintentionally connected two adjacent IC pins. But it was more common to find joints with insufficient solder or poor solder joints, where an IC pin was sort of resting on the PCB pad without actually bonding to it. Fortunately both problems were easy to fix with a soldering iron and a bit of flux.

 
Bad microcontroller chip

The onboard microcontroller trip is another potential source of failure. In my experience, these microcontrollers are pretty robust, and the only failures I have seen are caused in the field when customers accidentally connect the Floppy Emu cable backwards to their Apple II Disk II controller. This is distressingly easy to do, since the Disk II controller has bare pin connectors instead of a shrouded and keyed header. A backwards connection results in +12 and -12 volts applied to the mcu’s pins, killing it.

Unfortunately the symptoms of a bad microcontroller are nearly identical to the symptoms of a bad clock crystal: a completely unresponsive board, with no debugger activity. I had to review each non-responsive board’s history in order to guess which issue was at fault. In some cases I guessed wrong, and I ended up replacing the microcontroller, and then when that didn’t help, also replacing the clock crystal.

 
Other

The remaining issues were all one-offs. One board’s display header was physically broken and missing a pin, resulting in a blank display. Another board failed QA because the LED didn’t illuminate, except there was no LED! There was only a blank pad on the PCB. I also encountered a failure due to a PCB that was physically cracked, a long line running down the breadth of the PCB that could only be seen clearly under light from a specific angle. Two more boards defied my efforts to pinpoint the cause of their failures, and after spending too much time on them, I threw them into the scrap bin.

The very last board that I examined turned out to have no problems at all. I tested it extensively and it worked perfectly. This might have failed QA due to something external like a bad power supply or cable, or maybe it was simply miscategorized.

 
Final results

Of 68 Floppy Emus in the failure analysis heap, I managed to resuscitate 65 of them. That’s a pretty solid percentage! I learned to recognize the symptoms of certain failure causes, so I can address them faster if I see them again. More importantly, the failure analysis learnings (especially about clock crystals) will also help guide me in making design and assembly process changes, so I can reduce the number of future QA failures. Knowledge is power, as they say, and now I have lots of power.

Planning for the 2026 edition of Mactoberfest Meetup is well underway. This year’s event will take place on Saturday November 7, 2026 in Belmont California. We’re planning a glorious celebration of all things related to the classic Macintosh, with other connected vintage technology too. Don’t miss it!

The previous Mactoberfest Meetup was a hit, and this year promises to be bigger and better. We’ve got a few new things in store for attendees, including sponsors who’ve donated some cool hardware and tools, a Discord channel for pre- and post-meetup conversations between attendees, and more. Look for further announcements about these very soon.

With two months to go until the big day, there are currently 23 exhibitors registered, with more coming. We expect to welcome between 30-40 total exhibitors and several hundred attendees to this year’s Meetup. The next exhibitor to register could be you! If you have an old Mac, Apple II, or related gear, please do sign up as an exhibitor and share your collection. Your participation is highly encouraged, and Mactoberfest is intended to be a collaborative shared experience created by everyone who’s there.

Here are some highlights of what you can expect to find at Mactoberfest Meetup 2026 (with more still to come):

• A vintage Mac DIY repair station stocked with soldering tools, test equipment, and expert help.
• Exhibit of classic Apple and Macintosh networking equipment, highlighting connectivity across Apple II, 68K Macintosh, PowerPC, and early G3 systems.
• Collection tracing the history of Macintosh input devices, including mice and trackballs.
• Exhibit focused on magnetic storage for Macintosh computers, including internal and external hard disk and floppy disk drives.
• Collection of Clarus the Dogcow memorabilia and collectibles.
• After Dark screen saver exhibit featuring an original iMac running screen saver modules alongside a modern macOS emulator developed specifically for After Dark.
• Collection of unusual Macintosh systems, including a Mystic Color Classic with Apple IIe Card, a Power Mac 4400 prototype, and an Assistive Technology Freestyle.
• Mac-compatible handhelds, peripherals, and a hands-on setup for making lo-fi photo name badges.
• “The Satanicube”: an extensively modified and highly unusual Power Mac G4 Cube previously exhibited at VCF West and VCF SoCal.
• Intel Developer Transition Kit, a prototype computer designed to assist with the transition from PowerPC to Intel processors.
• Collection of Apple Newton handheld personal digital assistants.
• Collection of compact Macintosh systems, including 100-series PowerBooks, PowerBook Duos with docks, iBooks, and a PowerPC Mac Mini.
• Power Macintosh 7600 running non-Mac OS systems including Rhapsody, BeOS, and MkLinux, plus Macintosh IIsi running System 7.6, NetBSD, and Debian.
• Complete Power Computing system with original operating system and period promotional materials.
• Macintosh SE with custom MacEffects green case, Macintosh IIcx with Wombat USB accessories and games, Macintosh IIgs with Floppy Emu, and Power Mac G4 Cube with games.
• Macintosh IIfx with NuCF solid-state storage by zigzagjoe and Macintosh Portrait Display, plus a heavily modified Macintosh SE/30.
• Collection of compact Macintosh systems, including Macintosh 512K, Plus, SE, and/or SE/30.
• Macintosh Plus with custom display and miniature displays featuring After Dark screen saver modules.
• Macintosh SE with Voice Navigator, demonstrating early voice control of the Macintosh graphical interface.
• Macintosh IIci with a Caere OmniScan handheld scanner.
• Quadra 650 with PC DOS Card, PowerBook 2400c, graphite iBook, Macworld magazines, and 1990s Apple advertising.
• Macintosh SE running MacWrite and a connected ImageWriter II printer loaded with paper.
• Macintosh LC system in a custom neon orange case.
• Disassembled Macintosh 128K with exposed logic board, and all chips and ports labelled.
• NeXT computer hardware and software, the precursor to Mac OSX.
• Collection of 1980s and 1990s computer magazines including MacUser, MacWorld, and COMPUTE!
• Original Atari Pong home console from 1975.

For more details and to register for Mactoberfest Meetup, visit mactoberfestmeetup.org. You can also support Mactoberfest by contributing a few dollars to help cover our event costs. The meetup is free to attend, and we’re relying on your generous donations to help us pay for this thing. Thank you!

How do you go about troubleshooting a misbehaving microcontroller circuit? A few months ago I manufactured a new batch of Floppy Emu disk emulators. A number of them failed QA at the factory, with a set of symptoms that I’d never seen before in all my years of developing this device:

• Most of them simply wouldn’t boot up at all, despite verifying that power was good and the mcu was correctly programmed.
• Some exhibited “haunted” behavior, seemingly jumping to random sections of the mcu program code, outputting messages on the display that made no sense given the context.
• One of them appeared to work in slow motion, with LED blinking and display updates noticeably more sluggish than normal.

This was odd, to say the least. I have a lot of experience with the ATMEGA1284 microcontroller and the Floppy Emu circuitry that surrounds it, and I’ve become an expert at guessing what’s wrong based on the symptoms of misbehaving boards. These were all new and bizarre symptoms to me. Might they arise from different problems, or could they all point to one common underlying issue?

My intuition suggested some kind of systematic assembly problem. My contract manufacturer used a new subcontractor for this batch of Floppy Emu boards, so maybe a silent change to the process caused an unexpected issue? Parts substitution? Bad parts? Counterfeit chips? These QA failures sat in a pile on my desk for months, waiting for answers.

Probing, Poking, and Theorizing

Yesterday I finally decided to concentrate on the “won’t boot” devices, since that seemed like the most tractable problem. I put a few boards in a test harness, and connected power and a hardware debugger. The power supply voltages looked good. No obvious soldering problems were evident, but just to be sure I reflowed the solder on a few boards, without seeing any improvement.

On many of the boards, the hardware debugger could talk to the microcontroller and I was able to confirm the chip was correctly configured and programmed, but the program didn’t seem to actually run. At power-up the boards did… nothing. And with a smaller number of the boards, the debugger could not communicate with or even detect the chip. What could cause these symptoms? I brainstormed:

• Bad power. Seemingly ruled out by my measurements.
• Misprogrammed chips. I confirmed the configuration and reprogrammed several, without improvement.
• Bad chips.
• Chips stuck in reset.
• Clock problems.
• Problems with other circuit components (SD Card, CPLD, etc) causing electrical or program failures.

A batch of bad microcontroller chips seemed like the most likely explanation, so I desoldered the ATMEGA1284 from a board and replaced it with a new one from my stash. But after configuring and programming the chip, the board behaved the same as before, refusing to boot. That seemed to rule out problems with the chips themselves.

In the Floppy Emu program code, when the device first powers up, there’s some communication with the SD Card and the CPLD that happens before anything is drawn on the device display. I suspected that something might be going wrong during those steps, causing the program to freeze or crash and resulting in a blank display. To test this, I modified the program to blink the status LED twenty times as proof of life at the start of main() before doing anything else. Yes, with all the hardware tools at my disposal, I was back to caveman debugging with a blinking LED.

But there was still no joy, no LED blinking, no apparent program activity at all during power up. What the hell? Here I had a good microcontroller with good power, confirmed programmed correctly, in a circuit and board design that’s been in successful use for years. It wouldn’t even blink an LED. Since the blinking should have happened as the very first step of the program, its absence mostly seemed to rule out explanations related to failed interactions with other circuit components like the SD Card. So I focused in on the reset signal and the clock, the only two possibilities that I had left.

Clock Crystal Mysteries

Floppy Emu’s microcontroller uses an external 20 MHz crystal for speed and precision, but it also has an internal built-in 8 MHz oscillator. This particular board was still communicating OK with the hardware debugger, so for grins I tried changing the chip’s fuse configuration to select the internal 8 MHz oscillator as the clock source. Lo and behold, it worked! The device booted up and appeared to run normally, although obviously at only 40 percent of normal speed. I confirmed the same result with a few other boards – when I was able to get debugger communication and change the clock source to the internal oscillator, the board would boot. This wasn’t a fix, since the Floppy Emu won’t actually work correctly with an 8 MHz oscillator, but it was proof of major trouble with the external clock crystal.

If an external crystal isn’t working reliably, the microcontroller won’t have a reliable clock source. It will probably fail to run at all, or else act super glitchy. It will also cause problems with debugger communication. This all sounds a lot like my observed symptoms.

So let’s talk about this crystal oscillator circuit. Like almost all microcontrollers, the ATMEGA series has built in amplifier hardware to drive an external piezo crystal and force it to oscillate, using a circuit that I believe is called a Pierce Oscillator. I should know more about the theory of operation, but I’m mostly ignorant. What I know is that you connect the crystal’s two terminals to two ATMEGA pins using the shortest PCB traces that are practically possible, and add two external capacitors with values in the picofarad range, whose values are determined by a formula, and then everything works.

Investigating a bit further, I observed that all of the problem boards used a different crystal manufacturer than I have used previously. That’s fine, it shouldn’t have been an issue, but it seemed important given the circumstances. Previous editions of the board used this NDK crystal, but these troublesome boards substituted a similar ECS crystal. Both used the same physical footprint and advertised an 8pF load capacitance.

Speculations and Next Steps

As of today, that’s as far as I’ve gone with direct debugging, but I’m continuing to search for a smoking gun explanation. Maybe I got a batch of bad crystals? Possibly, and I can try reworking a board and replacing its crystal, but that explanation seems not very likely to me.

What about those two capacitors that form part of the oscillator circuit? Their values are important to the oscillator operation, and if the value is too far off from the optimal value, then the crystal won’t oscillate correctly or won’t oscillate at all. These tiny SMD capacitors bare no markings, so there’s no way for me to confirm visually that the capacitors are the correct ones. Maybe the subcontractor used the wrong value of capacitors on some boards? Speaking of which, what is the correct value?

Here we enter into a bit of Pierce Oscillator analog voodoo that I don’t understand very well. The correct value of the two external capacitors is given by the formula C_ext = 2 * (C_load – C_stray). C_load is the crystal’s load capacitance: 8pF in this case. C_stray is a measure of the stray capacitance of the microcontroller pins and PCB board traces. There’s no simple way to measure this directly, but for short traces on a two-layer PCB, I’ve seen estimates around 3pF to 5pF. Let’s call it 4pF. So C_ext = 2 * (C_load – C_stray) = 2 * (8pF – 4pF) = 2 * (4pF) = 8pF. In theory then, I should have two external 8pF capacitors paired with the clock crystal. In reality, the capacitors are 18pF.

18pF external capacitors. I don’t remember how I originally specified this value; it’s lost in the mists of time during Floppy Emu’s initial development phase. But looking at it again now, it certainly seems “not ideal”. The oscillator circuit can be fairly forgiving and the ATMEGA driver amplifier can work over a broad range of capacitance values, which is probably why I never noticed an issue before. But 18pF is not mathematically correct. My guess is that the oscillator circuit has been operating close to the margins, and now there’s something different enough about this ECS crystal, its ESR or stray capacitance maybe, that pushes the circuit far enough out of its comfort zone that it stops working entirely.

So now what? How can I confirm this theory and fix the issue? One possibility is modifying the ATMEGA’s crystal driver behavior by changing a fuse setting. I normally use the low-power crystal oscillator mode, which applies a driving voltage in the millivolts range, but there’s also an option for full-swing crystal oscillator. In theory this setting should work better in cases like this where the external capacitors are outside the optimal range of values. To test this, I altered the fuses on one board to enable the full-swing oscillator behavior, and… it didn’t work. The board still wouldn’t boot up, and it also stopped communicating with the debugger, so it’s now effectively a brick.

That leaves me with the possibility of reworking the boards and swapping the external capacitors for 8pF replacements. Or maybe 10pF or 12pF if I want to stay closer to the original design value, since problems can also arise if the value is too low as well as if it’s too high. Unfortunately my workshop doesn’t stock any appropriate capacitors in that range. I’ve ordered a variety of values to use for testing, so the conclusion of this mystery will need to wait until then. Stay tuned…

The Yellowstone Universal Disk Controller for Apple II computers is now back in stock! Thanks for everybody’s patience – manufacturing took much longer than expected.

Yellowstone is a universal disk controller card for Apple II computers. It supports nearly every type of Apple disk drive ever made, including standard 3.5 inch drives, 5.25 inch drives, smart drives like the Unidisk 3.5 and the BMOW Floppy Emu’s smartport hard disk, and even Macintosh 3.5 inch drives. It combines the power of an Apple 3.5 Disk Controller Card, a standard 5.25 inch (Disk II) controller card, the Apple Liron controller, and more, all in a single card.

Features

• Add 3.5 inch drive and Smartport hard disk support to your Apple IIe or II/II+
• Provide more disk connectivity options for your Apple IIgs
• Bring Macintosh 3.5 and naked Apple 3.5 inch drive mechanisms to the Apple II
• Drop-in replacement for an Apple Liron controller card (with optional DB-19F adapter)
• Drop-in replacement for a standard 5.25 inch or Disk II controller card
• Run two drives of different types on twin independent disk connectors
• Disk II controller emulation mode for tricky copy-protected disks
• Works with DOS 3.3, ProDOS, GS/OS, and more
• User-upgradable firmware for future feature enhancements
• 20-pin ribbon cable connectors or optional 19-pin D-SUB connectors

You’ll find the Yellowstone controller here in the BMOW Store. For more details, please see the product description page.

For everyone in the San Francisco Bay Area, Mactoberfest Meetup is back by popular demand! After a successful event in 2023, we took a few years off, and now we’re returning on November 7 to the same historic church venue in Belmont where we held the previous meetup. You’re invited to join us for a day of fun and the opportunity to show off your Mac and Apple gear. Come relive the glory days of Hypercard, desk accessories, and flying toasters! If you don’t live in California, steal a car and drive here. If you’re in jail, break out! You don’t want to miss this.

Here’s a photo from the 2023 meetup to give you a sense of the event:

What to expect at Mactoberfest Meetup:

• Rooms full of vintage Macintosh fanatics and crusty old computers
• Flea market – Items can be offered for sale at display tables or in the “marketplace corner”.
• Freebies table – Got anything you can give away for free? Put your donations of extra disk drives, cables, adapters, and peripheral cards here. See something you like? Take it!
• Workshop table – Stocked with soldering irons, tools, a multimeter, and maybe an oscilloscope or logic analyzer. Troubleshoot your broken computers here, or build something new.
• LAN game competition – How about Spaceward Ho at 3:00 PM? Start practicing now!
• Spontaneous ideas – System 6 trivia quiz? 2400 baud modem demos?

How you can help:

Can you take a turn for an hour at the workshop table, helping somebody to troubleshoot a Sad Mac error code or recap a motherboard? How about organizing the LAN tournament? Or if you have another idea for a fun activity, great! Your willingness to help is the critical ingredient for the meetup’s success.

What you should bring:

• Vintage computer hardware, media, and collectibles.
• Extension cords and power strips for your computers – We definitely won’t have enough for everybody.
• Items to sell in the flea market.
• Soldering irons and tools like tweezers, cutters, and magnifiers. Multimeters and test equipment. Capacitors and components for common repairs.
• Odds-and-ends to donate for the Freebies Table – How many old SCSI drives and spare motherboards do you really need, anyway? Stop hoarding them.
• Kits and DIY stuff – Can you donate anything that ‘just needs assembly’, like a microcontroller kit or a set of replacement floppy drive gears?
• Cool stuff you can donate as prizes or give-aways.
• Snacks, water, or drinks – Donations will be very appreciated.

For more details and to register for the event, please visit mactoberfestmeetup.org

MACTOBERFEST
Bay Area Classic Macintosh Meetup / Demo / Workshop / Swap-Meet / Tournament / Whatever
Saturday 7 November 2026, 11:00am to 5:00pm
Belmont, California
(event address is on the registration form)

Please don’t forget to RSVP if you’re if you’re maybe, probably, or definitely planning to attend. This will help us keep track of who’s bringing what items, and the likely overall attendance level.

Questions, comments, suggestions? Hit me! This is a meetup, not a VCF clone, and its success depends on everyone’s participation and involvement. If you’ve got an idea for something that you’d like to see, come join us and make it happen!

mactoberfestmeetup.org