BMOW title
Floppy Emu banner

Garbage In, Garbage Out

I’m beginning to suspect there’s something subtly wrong with my Apple IIe, which might explain a lot of strange intermittent errors I’ve recently observed with Yellowstone and Floppy Emu development. It’s hard to know what to conclude, but when you can’t trust your test equipment, it’s impossible to trust the validity of your test results.

Recently I used this Apple IIe to test my new Floppy Emu OLED prototype board. When connected to a standard Apple Disk 5.25 controller card, everything worked fine initially. But when I tried doing some ProDOS file copy operations, the Floppy Emu spontaneously reset to the happy face / self-test screen. DOH! I tried it twice, and the Emu reset itself during the copy operation both times. I concluded there must be something wrong with the prototype board.

Then I tried the copy test two more times, and got different results. The Emu prototype didn’t reset itself, but the OLED display went blank several times during the copy operations. Hmmm.

So then I tried a plain vanilla Floppy Emu Model B, the same hardware and firmware that I’ve been using successfully for more than a year. I found that when trying to boot the Apple IIe from a ProDOS v1.9 disk image, the Model B’s LCD went blank several times during booting. Huh? This happened in two consecutive test runs, but then mysteriously stopped happening. I also tried the same file copy operations I’d done with the OLED prototype board, and saw a similar behavior where the LCD went blank a few times during the copy. But as before, after reproducing the bug twice in a row, it stopped happening.

Finally I went back to the OLED prototype board, and this time everything worked fine. No more unexplained resets or display blank-outs.

Maybe there’s something wrong with the Apple IIe’s power supply, or some problem where it needs to warm-up for a while before it works reliably? My first OLED prototype board tests were the first time I’d powered on the Apple IIe in several days, so it was cold. During an hour of testing, the strange Floppy Emu problems I’d observed gradually disappeared. It doesn’t really make sense to me, but it’s the best explanation I can think of. This might also explain some strange unexpected resets of the Floppy Emu last month, when I tested it with the Yellowstone card. In fact, it casts doubt on all of my Yellowstone testing.

Read 8 comments and join the conversation 

Floppy Emu OLED Prototype

Displays, displays! I’ve built a prototype Floppy Emu board that uses a 1.3-inch 128×64 OLED display, instead of the existing 1.4-inch 84×48 LCD. It works, and it looks very nice. So is this the answer to my display worries? Take a look at the photos, and tell me what you think.

Why change displays? The current edition Floppy Emu uses an 84×48 LCD that’s a clone of the Nokia 5110 mobile phone display. It’s a decent module with a nice built-in backlight, but the displays have caused me a lot of headaches over the years due to their iffy quality control. Around 10-15% are dead on arrival and must be discarded during manufacturing and testing. A large fraction of the remainder exhibit flaky behavior until the LCD’s pressure-fit connector is manually fine-tuned. These LCD hassles consume too much of the manufacturer’s time and my time. The supply chain for the 5110 LCDs is also problematic, coming only from small eBay and Aliexpress sellers, instead of a major manufacturer who can provide documentation and support.

Last month I reviewed some potential replacement displays. There weren’t many great options, but after lots of consideration I settled on this 1.3-inch OLED. It’s similar to the 5110 LCD in many ways, being a 1-bit graphical display with an SPI interface. The physical size is a bit smaller than the 5110 LCD, but they’re close. Unfortunately the supply chain situation for the OLED is even worse than the 5110 LCD, and the cost is at least $2 more per unit. But if I can save myself $2 worth of hassle, it will be worth it. A few weeks ago I hacked up a Floppy Emu board to support the OLED, just to prove that it worked, and today I now have a proper OLED Emu prototype.

I For One Welcome Our New OLED Overlords

The first thing you’ll notice about the OLED is that the contrast is amazing. The display is bright and very crisp, and the photos can’t do it justice. I never thought the 5110 LCD looked bad, but the OLED looks far superior.

The extra resolution of the OLED helps a lot. Text characters on the 5110 LCD are 3×6, while on the OLED they’re 5×7. This makes possible a more finely-detailed font that’s a nice improvement in legibility.

Text on the OLED is smaller, and at first I was concerned this would make it difficult to read. But with the awesome contrast and better font, my weak eyes find the OLED to be about equally easy to read as the larger text on the 5110 LCD. My wife declared the OLED more readable than the 5110 LCD. The OLED text is also larger than the text on my Backwoods Logger Mini (2011 flashback alert), and is about the same size as this CorelDRAW quick reference card I happened to have sitting out:

The OLED shows eight rows of text, compared to six rows on the 5110 LCD. This is a nice bonus when scrolling through a long list of filenames. With some extra development work and possible resulting display update slowness, future firmware might be able to trade fewer rows for larger text, but the OLED is designed around eight rows in hardware.

Fun with Fonts

I tried two different fonts. The first font has most letters being 5×6 pixels, with only the descenders of g, j, p, q, and y dropping below the baseline to make a 5×7 letter. Since each row is 8 pixels tall, the 5×6 letters provide two pixels of whitespace between rows. Here’s a sample:

The second font has most letters being 5×7 pixels. There are no true descenders, so g, j, p, q, and y sit on the same baseline as all the other letters. Because the letters are one pixel taller, this font is slightly more detailed and more readable than 5×6. But with only a single pixel of whitespace between rows, the text looks sort of cramped. I decided I didn’t like it, but maybe you’ll feel differently. Here’s a sample:

For comparison, here is the 3×5 font on the 5110 LCD:

Now What?

So it works. Now what? If I want to move forward with this OLED, there’s a lot of work to do. I need to find a supplier who can reliably provide hundreds of these displays. I need to get my updated PCB to my manufacturing partner. Because it’s a new design, I’ll have to pay them a bunch of money for one-time engineering fees. I’ll need to update the manufacturing instructions, and the test procedure, and the firmware that goes with it. And I’ll need to design a new case to fit the new board and OLED. It will all be a lot of work, and when it’s done my costs will have increased by $2 per unit. Is the pain worth it? Probably yes, but I’m still undecided.

I might do some experiments with another display I recently discovered, this COG 12864 LCD. It’s a little bigger than the OLED, which might be helpful, but it lacks the OLED’s excellent contrast and crispness. Its resolution is the same as the OLED, and it’s roughly the same price, and the manufacturing work needed to change displays would still be the same. I’m not sure there’s a compelling reason to choose the COG 12864 LCD over the OLED, but it’s a similar contender. Hmmm.

Of course the best option might still be to do nothing, and keep the 5110 LCD. Yes it’s a headache, but it’s a headache I’m familiar with and have dealt with successfully for four years. A new display would bring its own set of headaches through required manufacturing changes and case redesign, so I’m going to have headaches no matter what! But when I think about the best long-term solution, I’m drawn to the idea of a more reliable, better looking OLED.

Enjoy some gratuitous unnecessary zoom:

Read 11 comments and join the conversation 

Floppy Emu Display Experiments

Last week I wrote about my troubles with the Nokia 5110 LCD used by Floppy Emu, with a discussion of possible replacements. There was no clearly obvious alternative, as most of the options were too big, too small, too expensive, not available from a reliable supplier, or had some other shortcoming. After some thought, I settled on the 1.3 inch OLED as the most likely alternative. And now, after hacking away on the Emu’s firmware, I have a working example of a Floppy Emu with this 1.3 inch OLED display.

Nokia 5110 Clone LCD: 1.4 inch diagonal display, mounted on a carrier PCB, 84×48 pixels, SPI interface, 1-bit graphics

No-name OLED: 1.3 inch diagonal display, mounted on a carrier PCB, 128×64 pixels, SPI interface, 1-bit graphics

The photo shows a head-to-head comparison between the old and new displays. The picture doesn’t do the OLED justice – although it looks blurry here, it’s actually very crisp and pleasant to look at. The only significant problem is the tiny pixels, which makes the text a bit difficult to read. Although the display diagonal isn’t much smaller than the Nokia LCD (1.3 inch vs 1.4 inch), the aspect ratio is different, and the OLED’s height is only 2/3 the Nokia’s height. Then the OLED packs 1.33x as many vertical pixels as the Nokia, so the OLED actually has 2x as many dots per inch. You can see in the photo, when drawn with the same number of pixels, the OLED text is only half as tall as the Nokia text. Get out your reading glasses.

It’s not too bad, and I could probably live with text this size, but it’s not ideal. It would be great if I could simply make all the text 1.33x taller to take advantage of the OLED’s extra resolution, but unfortunately it’s not that simple. Text needs to be a whole number of pixels tall, and with the Nokia LCD there are six rows of 8 pixel tall text. The most I could increase this with the OLED would be 10 pixel text, 1.25x taller, for 60 pixels total, leaving 4 pixels wasted. But 10 pixel text is an awkward number, because the OLED’s command interface is built around the concept of “pages” that are 8 pixels tall. If text spans a fractional number of pages, I’ll need to completely redo the way display updates are performed, and maintain a framebuffer of the whole display area. Unless I get really fancy (dirty rectangles anyone?), that will require re-sending all 128×64 pixels of the framebuffer to the display every time I draw anything. That will be noticeably slow, which is one of the things I wanted to avoid.

So instead of making the text taller, I’ll probably make it wider. That should help a lot with readability. With 6×8 pixel characters, I’ll get 8 rows of 21 characters each, compared to 6 rows of 21 4×8 pixel characters on the Nokia LCD. Two extra rows of text will be nice.

I still need to look at the power consumption of the OLED as compared to the Nokia LCD. When I have two Floppy Emus powered by a single 5V USB supply, the one with the OLED glitches and resets whenever I turn on the one with the Nokia LCD. I’m not sure what’s causing that, but it doesn’t happen with two Nokia LCDs.

The final hardware headache will be modifying the Floppy Emu PCB layout. In order to keep the OLED centered, I need to relocate the LCD header into a spot that’s currently packed with traces and chips. I’ll also need to move the header from the bottom section of the board to the top (unless I mount the OLED upside-down), which means running several new traces across the full height of the PCB, or just redoing the entire board layout, which doesn’t sound fun. I’ll probably want some kind of mechanical support for the bottom end of the OLED too, since the OLED only has pins along its top, and it tends to hinge downward from those pins by force of gravity. That’s not a deal-breaker, but it looks a bit unattractive.

There will be some software challenges to address as well. I can’t just replace all the Nokia 5110 code with OLED code, because then future versions of the Floppy Emu firmware won’t work on all the Emus out in the world now with Nokia LCDs. I don’t want to maintain two different versions of the firmware either, one for each display type. Ideally I’ll find a way to create a single firmware that knows how to control both types of displays, and can dynamically detect which one is present.

The OLED display costs about twice as much as the Nokia LCD, but it’ll be worth it if it eliminates the headaches I’ve had with the Nokias. Right now I don’t expect I’ll need to change the price of the Floppy Emu. I’ll see how things look once everything is finalized.

Read 6 comments and join the conversation 

More Yellowstone Trouble

I did some surgery on the Yellowstone board, in an attempt to address major problems with power supply noise and data bus overshoot, previously discussed here and here. I replaced the 74LVC245 that drives the data bus with a 74LVC8T245. That’s a dual-supply chip that some readers suggested earlier: it ensures the bus outputs will drive all the way up to 5V, while hopefully reducing any noise coupled to the 3.3V supply, and avoiding the possible violation of limiting values with the 74LVC245 that I mentioned in my last comment to the previous blog post. The Yellowstone board still mostly works with the 74LVC8T245, but now the data bus overshoot climbs to a whopping 9 volts! Arrrrgh.

Channel 1 (yellow) is a copy of the Apple II slot’s /IOSELECT signal, passed through the FPGA. When it goes low, it means the Apple II wants the card to drive the bus.
Channel 2 (light blue) shows D0 on the data bus, with a nasty overshoot.
Channel 3 (pink) is an internal active-high debug signal from the FPGA that shows when it’s outputting a value for the 74LVC8T245 buffer. This is a sanity check on what’s happening.
Channel 4 (dark blue) shows the Yellowstone card’s ground, with respect to the Apple II system ground. Note there’s about 1V of peak-to-peak ground noise.

Something is badly wrong, and I can’t find it. I could design a new board with some series termination resistors, as a few people suggested, but my intuition is that isn’t really the main problem. None of the other Apple II cards I’ve examined appear to use any termination at all. 9V is a massive overshoot. And termination issues wouldn’t explain the problems observed on the card’s 3.3V power and GND supplies. I feel like I’m headed down the wrong path.

I’ll probably ice this project for a while, since enthusiasm for further debugging has run out. On to something else…

Read 9 comments and join the conversation 

Quest for a Decent LCD

Floppy Emu uses an 84×48 graphical LCD display. It’s just a low-resolution 1-bit display, but it’s fast and easy to use, and has a nice built-in backlight. The display is actually a clone of the old Nokia 5110 phone display, and it’s made by semi-mysterious third-party factories in Shenzhen. It can be purchased in bulk for about $2.50 apiece. Photo from RH Electronics.

The big problem with these Nokia 5110 displays is that their reliability stinks. The actual LCD module (the glass and metal bit) is clipped onto a supporting PCB with some passive electronics, and it’s only a pressure-fit holding the two together. If it’s not clipped in just right, the display will exhibit contrast problems, or glitchy behavior, or just won’t work at all. Gently pushing on the LCD frame sometimes changes the pressure-fit enough to make these problems appear and disappear. Adjusting and tightening the LCD clips, as described in the Floppy Emu manual, is the only thing I’ve found that helps.

The electronics assembler that builds Floppy Emus must go through every LCD to check for problems. They usually end up discarding about 10% of all the LCDs, because they don’t work no matter how the clips are adjusted. Once the boards are finished, I do a second check of each LCD immediately before it’s shipped to the customer. This often requires more fiddling with the clips, or manual contrast adjustments, and a further 5% of LCDs are discarded. It’s very time-consuming, but despite all this effort, some troublesome LCDs still reach customers who must then make further adjustments.

In the most recent batch of LCDs, the pressure-fit contact design changed slightly, and it now appears to be even more troublesome than before. At the same time, the LCD bezel was unexpectedly enlarged by 2mm, forcing me to redesign the Floppy Emu acrylic case to match. This is a risk of buying generic parts from eBay and Alibaba, with no manufacturer to stand behind them or datasheet to document them.

Surely There Must Be Something Better?

It would be very nice to replace the 5110 displays with something similar but more stable. A replacement would need to handle about 84 x 48 1-bit pixels (equivalent to 21 x 6 text characters), with a diagonal size about 1.5 inches, and ideally use an SPI interface. Unfortunately, I’ve found nothing that even comes close. The alternatives are either much too large/small, lack graphical capabilities, are too slow, or are much too expensive.

Character and numeric displays aren’t appropriate, since they can’t do graphics or six rows of text. So looking at Digikey’s Display Modules – LCD, OLED, Graphic category, and sorting by unit price quantity 100 purchasing, and including only those results that have at least a few hundred units in stock, I found these:

128×128 RGB LCD, 1.44 inch diagonal, $4.23 – This could almost work, except I believe it’s a 24-bit color display, so the microcontroller would need to move 24x as much data to draw on it. And because it’s a much higher resolution, the amount of data to moved must be still higher to maintain the same font sizes as the old display. And it’s a slow I2C interface, instead of fast SPI. And there’s no datasheet.

128×32 LCD, $8.01 – This is an odd shape, doesn’t have enough vertical resolution, and uses a parallel 8-bit interface.

Another 128×32 LCD, $8.69 – Also an odd shape, and not enough vertical resolution.

128×160 RGB LCD, 1.8 inch diagonal, $8.83 – This is another color, higher-resolution display like the $4.23 one, but it uses a parallel interface.

128×64 LCD, 2 inch diagonal, $9.52 – Too big, too expensive, uses a parallel interface.

Non-Branded Options

Nothing from DigiKey looks suitable. What about other options from eBay or Alibaba?

128×64 OLED, 0.96 inch diagonal, $2.91 – This could sort of work, and I have one of these modules already. But it’s tiny, smaller than a postage stamp, which isn’t really suitable. It’s also I2C which means the interface is comparatively slow. Coming from a random non-branded eBay seller, it’s also not clear it would be any more reliable than the LCD display I have now.

128×128 RGB LCD, 1.44 inch diagonal, $3.04 – This is basically the same as the $4.23 module from DigiKey. Although this one says it has an SPI interface. Maybe this is the best option from a list of not-so-great alternatives.

Read 11 comments and join the conversation 

Yellowstone Arrgh

I’m still struggling with these Yellowstone card electrical problems, but getting nowhere. Very frustrating. I’ve focused my attention on the large amount of overshoot on the data bus, and the power supply fluctuations, under the theory that minimizing those will eliminate all the functional glitches and failures I’ve experienced.

The FPGA has eight bidirectional pins that are connected to one side of a 74LVC245 buffer. The other side is connected to the data bus. The buffer’s direction is controlled directly from the CPU’s R/W signal, and the buffer’s output enable is controlled by FPGA logic. The FPGA pins direction are controlled by separate FPGA logic. See the schematic here.

The image above shows the moment when the card starts to drive a value on the data bus.

Channel 1 (yellow) is a copy of the slot’s /IOSELECT signal, passed through the FPGA. When it goes low, it means the Apple II wants the card to drive the bus.
Channel 2 (light blue) shows D0 on the data bus. Notice how it climbs from 0 to 2V, retreats back to near 0, shoots up to 4.6V (a good trick for a 3.3V chip), and finally settles at 3.3V? Yuck.
Channel 3 (pink) is an internal active-high debug signal from the FPGA that shows when it’s outputting a value for the 74LVC245 buffer. This is a sanity check on what’s happening.
Channel 4 (dark blue) shows the Yellowstone card’s ground, with respect to the Apple II system ground. Note the scale: it doesn’t look like much, but there’s almost 1V of peak-to-peak ground noise.

Besides the data bus overshoot, there’s also something odd with /IOSELECT on channel 1. Because there’s no convenient place for me to physically probe the actual /IOSELECT signal, I used an FPGA debug output pin to mirror /IOSELECT and capture it with the scope. The result should look identical to the actual /IOSELECT, except shifted to 3.3V logic levels rather than 5V, and with a few nanoseconds of extra delay. But why is there 1V of undershoot on the scope when /IOSELECT first goes low? The scope is capturing a debug output that’s just a two-inch trace on the Yellowstone card, with no other load except the scope probe. I was not expecting to see any undershoot or overshoot on these debug signals, and the channel 3 (pink) debug signal doesn’t show any.

As bad as that scope trace looks, this one looks worse:

This looks like some kind of glitch, because the FPGA is only outputting for about 60ns, but a normal output lasts about 500ns. /IOSELECT starts to fall, but then goes back up, and shoots up to 5.0V (remember this is an output from a 3.3V chip). The data bus voltage spikes to 6.6V, and the Yellowstone ground reaches 2.0V relative to Apple II ground. Arrgh!

In an effort to troubleshoot this glitch further, I switched channel 2 (light blue) to show the direction of the Apple II motherboard’s data bus buffer, which should (I think) simply be the CPU’s R/W signal. This was the result:

It appears that the glitch occurs just before the end of a CPU write, when the direction switches from low to high. Assuming that’s the CPU R/W signal, that means the glitch starts during a time when R/W is 0, which should be impossible. Here’s the Verilog equation for the channel 3 (pink) signal:

assign isOutputting = (rw && ~_romoe);

If R/W is zero, then isOutputting must also be zero, but from the scope trace it’s obvious that’s false. There’s something happening here that I don’t understand. Maybe the Apple II motherboard’s data bus buffer direction isn’t driven from the CPU’s R/W, but from some other buffered or computed signal.

When I rearranged some things in an attempt to investigate the glitch further, it disappeared. I was never able to find an explanation.

Head Scratching

There are so many strange unexplained problems, it’s a miracle that the card actually works most of the time!

So what causes big overshoots and supply fluctuations? I can think of a few possibilities:

1. 3.3V voltage regulator can’t meet the instantaneous demand for current
2. Insufficient bypass capacitors on the card
3. 5V and GND connections from the card to the slot are too wimpy, and can’t carry enough current without significant resistive loss
4. Impedance mismatch on the data bus, signal reflections, etc.

Considering each of these in turn:

1. There’s a 300 mV LDO on the card, which I’ve calculated should be enough for the average demand. Brief spikes in demand should be satisfied by the capacitors. What’s the behavior of a typical LDO when it’s overtaxed? Does the regulated voltage drop, or does ground get pulled up, or both? If I make another revision of the Yellowstone board, I’ll substitute a larger regulator just to rule out this possibility.

2. My intuition pointed to this explanation first, but it hasn’t been confirmed by experiments. You can see all the bypass capacitors in the schematic: at least one 0.1 uF ceramic cap for the power supply on each chip, plus others recommended by the FPGA and LDO datasheets. I tried jury-rigging additional capacitors at various points on the board, but they didn’t make an observable difference in the signals as viewed on the scope. And even if this were the problem, I could imagine it causing the overshoot seen on the databus, but I can’t see how it would cause Yellowstone’s ground to rise to 2.0V relative to Apple II ground.

3. I tried connecting some extra jumper wires from ground points on the card to ground points on the Apple II motherboard. It didn’t make an observable difference in the signals as viewed on the scope. But maybe 5V has a problem?

4. An impedance mismatch could be part of the problem, but what would I do about it? The Apple II bus can contain anywhere from zero to six other cards of assorted types, so it’s not clear what impedance I should try to match. And all the vintage 1980’s cards I’ve observed don’t appear to have any termination resistors or other obvious impedance matching circuitry – they just connect a 74LS245 or similar buffer chip directly to the bus.

For clarity, I don’t think Yellowstone’s use of a 3.3V 74LVC245 is a problem, at least not in the way suggested by some commenters. Yes, this chip can only drive to 3.3V, but that’s plenty high enough to register as a logical high for the 74LS series logic used elsewhere in the Apple II. And the scope traces clearly show that the data bus voltages are high enough. If the 74LVC245 is causing a problem, it must be some less-obvious mechanism like coupling noise to the 3.3V supply, or maybe a too-fast slew rate.

I’ve pounded my head long enough, and am running out of ideas. Debugging is hugely impeded by the absence of any place to physically probe many of the bus signals, and by a scope with only four analog channels. The scope also has a 70 MHz bandwidth, which might be a factor. I feel too much like I’m scratching randomly in the dark, rather than making systematic progress towards narrowing down the causes of the problems.

My temptation is to design a new revision of the card, and include some speculative changes that might help. I could substitute a beefier LDO, add more capacitors, make the power connections extra-robust, and include additional debug headers for probing bus signals. It can’t hurt, but realistically it probably won’t help either. I could also try adding some termination resistors, but that just seems wrong somehow, given that no other Apple II cards appear to use them.

Electronics can be hard. In the 10 years I’ve been writing this blog, I’ve never encountered a problem that had me so completely stalled as this one.

Read 16 comments and join the conversation 

Older Posts »