BMOW title
Floppy Emu banner

Archive for the 'Business' Category

Semiconductor Shortage and Business Threat

I’ve written several times about the global semiconductor shortage and the difficulty of getting parts. This has been a problem for everyone since COVID hit, but a year ago I’d hoped things would be improving by fall 2022. Unfortunately it’s worse than ever. It’s now looking increasingly likely that I’ll need to cease most production sometime in 2023, sell off the remaining stock of hardware, shut down the BMOW business, and ride off into the sunset to do something else.

Is it really that bad? In the pre-COVID days, I could just buy thousands of parts from Digikey or Mouser whenever I needed them. More recently I’ve had to place orders for chips 3-4 months in advance, which requires more upfront planning but is still doable. But now several key chips for Floppy Emu and Yellowstone just aren’t available with any realistic lead-time. Suppliers are quoting dates in 2024, or just declining to give any dates at all. Yellowstone has been in this zone for a while, but now Floppy Emu is falling into the same trouble, and without those two products I really don’t have a business.

Yellowstone uses a Lattice FPGA that was widely available when I designed the board, but has since become unobtanium. Officially they’re still quoting delivery dates in late 2023, but the dates keep moving back. I strongly suspect that the dates are based on nothing more than hopes and prayers. A few days ago I received an unexpected email from Mouser, telling me that the FPGA was now back in stock and available to order. But I checked the site, and it still showed zero stock and a year+ lead time. A Mouser rep confirmed this was an error. Ugh.

For the Floppy Emu, microcontrollers from Atmel (now Microchip) have been harder and harder to find. But at least the lead times are somewhat manageable, currently about 4 months, and so far they seem to be hitting those dates. The biggest challenge is the CPLD from Xilinx (now AMD), which isn’t available from any official supplier and where reliable delivery dates are scarce. I had some Xilinx parts on order from Mouser with an expected delivery date of January 2023, then I received notice today that the delivery date has been pushed out to January 2024. Say what?

Some of these parts may be available from third-party suppliers, but I hate dealing with these guys. The whole surplus parts industry seems built on a foundation of deception and lies. Some of them will give an upfront quote if you ask nicely, but many use a retroactive repricing approach that drives me bananas. They might advertise having 37957 parts in stock with a price of $6.02 each, but the fine print says “Due to the shortage of the global supplier, some products may rise in price and out of stock. We will manually confirm after placing the order.” So I place an order and pay for it, then 24 hours I get an email saying sorry we only have 233 parts and the price is $25.44 each. If I decline, then they (eventually) refund me, but it’s a huge waste of time and a terrible way to do business. Even if I can get the parts, I’ll pay through the nose and my retail product prices will surely need to increase.

I’m afraid it’s a gloomy forecast. I’ll keep scrounging for parts as long as I can, and hope that this shortage eases soon.

Read 10 comments and join the conversation 

UPS International Shipping Redlining in South San Francisco

I’ve stumbled across a puzzling mystery. If you live in the southern part of the city of San Francisco, or in a large area further south into San Mateo County, UPS will not sell you international shipping labels. But change your Ship From zip code to a different city just a few miles away, and UPS will be happy to quote you for international shipping. I reached this conclusion after spending many hours pricing dummy international shipments with different Ship From and Ship To addresses, package sizes and weights, and content types. Only the Ship From address seems relevant. A UPS agent confirmed this behavior but couldn’t offer any explanation or solution.

I first discovered this problem on April 4, while using the third-party shipping service Pirate Ship to generate some UPS international shipping labels. For all my packages, Pirate Ship offered US Post Office shipping options, but nothing from UPS. After several hours working with Pirate Ship’s tech support, they concluded UPS was declining international shipping service to this region for some unknown reason. They told me to take up the issue with UPS.

Initially I didn’t believe Pirate Ship’s explanation, but then I discovered I could reproduce the same problem directly at the UPS.com page for international shipping quotes. You can try this too. Go to the UPS international shipment cost calculator page, as a guest (not logged in). Enter these package details:

Ship From: United States, Redwood City, zip code 94065. Or mostly anywhere else in the United States, except one of the redlined areas listed below.
Ship To: France, postal code 75008, Paris. Or try any another major city in Europe, Asia, wherever, it doesn’t matter.
Customs Value: 119
Package Details: My packaging, 11 x 9 x 2 inches, 0.8 pounds, 119 dollar value

You’ll see a list of available UPS shipping methods with specific price quotes. For example, UPS Worldwide Expedited service costs $154.44 to Paris. You’ll get the same behavior with Ship From addresses in most parts of California or elsewhere in the United States.

Now try the same experiment again, but for the Ship From location, choose one of the following cities and zip codes. These cities are a contiguous geographic region just south of San Francisco, centered around the San Francisco airport. The total population is about 1 million people.

94002 Belmont
94403 San Mateo
94402 San Mateo
94401 San Mateo
94404 Foster City
94010 Burlingame
94030 Millbrae
94128 San Francisco
94112 San Francisco
94132 San Francisco
94066 San Bruno
94080 South San Francisco
94019 Half Moon Bay
94044 Pacifica
94015 Daly City
94005 Brisbane

When you repeat the test using one of these Ship From addresses, UPS.com complains:

The shipment descriptions you have indicated may require special assistance. Please check the accuracy of your request or contact your local UPS office for assistance.

This is consistent with the apparent UPS international shipping redlining that I observed when using Pirate Ship. I spoke with the staff at the local UPS Store, but they’d never heard of this problem, and said their outgoing international shipments were working just fine. Through trial and error, I discovered that Shopify Shipping isn’t affected by this problem either, so it may have something to do with the class of UPS shipping account used to buy shipping labels. The problem definitely affects Pirate Ship, UPS.com’s own shipping quote system, and probably others too.

I then spent a long time on the phone with other UPS support agents who told me I was wrong, and that the Ship From zip code didn’t matter for determining international shipping service availability. On the third try, I was finally able to demonstrate the problem to a UPS agent, but she could only say “I don’t know why it’s doing that”. She couldn’t offer any explanation or solution. That was several days ago, and the problem is still happening today.

Read 5 comments and join the conversation 

BMOW Business Update – Good, Bad, and Scary

It’s been a while since I last shared any business updates for BMOW. The past few years have been an exciting trip, as my personal technology blog has grown into a side hustle and then into a bona fide business. Initially BMOW was something I did in my spare time away from my real job (video game development), then it was something I did while I was between jobs, and now for the past few years it has been my real job. The experience has been great, but now the global chip shortage has scrambled everything, and the future outlook is growing scary.

 
The Good

First the good news. The gradual transformation from blog to store began in 2014, when a few blog readers started asking to buy the Macintosh floppy disk emulator that I’d been writing about. I sold about $18000 of hardware that first year, doing all the assembly and fulfillment by hand. It didn’t leave a huge profit after subtracting all the cost of parts, tools, and other business expenses, but it was a thrill to be making something that people wanted.

In the time since then, sales have grown consistently every year. I’ve moved the assembly work to dedicated manufacturing partners, added many new products to the BMOW line-up, and even hired a part-time employee to help with order fulfillment and customer support. In 2021 I sold several thousand Floppy Emus, plus lots of other products too, which was amazing. Gross revenue was into the hundreds of thousands. My net income after all expenses is less than I could earn as a software developer here in Silicon Valley, but it’s still a respectable number, and being my own boss is fantastic. I couldn’t imagine going back to a regular full-time job now. Sure, I’m always complaining about the challenges of international shipping or payment processing disasters or something else, but that comes with the territory of running my own business.

 
The Bad

If there’s anything bad here, it’s the overall fragility of the business. A great business would have revenue from many diverse products, with reliable sources of supply, and a large and growing market. I have none of that. The market is owners of obsolete computers from 25+ years ago. There simply aren’t many of those, and the number is shrinking every year as some computers break and can’t be repaired. How many working Apple II and classic Macintosh systems even exist today? Ten thousand? Ten million? I really don’t know. I keep thinking that I must have saturated the market already, but it hasn’t happened yet.

Product diversity is another weak point. BMOW sells eighteen different products, but 80 percent of revenue comes from the Floppy Emu Disk Emulator. Everything else is basically just a distraction. If anything threatens the supply or sales of Floppy Emus, then the business is in big trouble.

 
The Scary

Friends, the global chip shortage is very bad. For over a year columnists keep predicting it will get better soon, but it only gets worse and worse. This won’t continue forever, but how much longer will we have to endure this crunch? Six months? A year? Two, or three? And what will the landscape look like then? Will semiconductor makers take this opportunity to discontinue some of their oldest and least-profitable chips, the chips that are needed for vintage computer hardware?

The Floppy Emu uses two primary chips: a microcontroller and a programmable logic device. The mcu is an Atmel ATMEGA1284 and the logic device is a Xilinx XC9572XL. Both are relatively old technology, like 15+ years old. Both have seen their prices double or triple in the past couple of years. And both now have very limited or zero stock everywhere, with little hope of getting more stock anytime soon. Maybe I can get more in six months, or a year? Maybe never?

The last time I did a Floppy Emu manufacturing run, it was very difficult to find parts, particularly the ATMEGA1284. I almost gave up. My manufacturing partner made a lot of calls, and eventually found a surplus parts broker who could fill the order for a nosebleed price. As of today, I have enough finished Floppy Emus to last until roughly September at my normal rate of sales. I don’t know what happens after that. If I can’t find the required parts, or redesign the board around new parts, then I’m basically out of business.

 
Strategizing

What would you do in this situation? I can’t decide, and I’m spinning my wheels trying to decide what to do, while the clock is ticking and time is running out.

One option is to redesign the Floppy Emu hardware around new parts with better availability. The existing parts are pretty old anyway, so a refresh would make sense. But what new parts should I choose? It’s not just the ATMEGA1284 that has supply problems – it’s virtually all microcontrollers from Microchip, ST, TI, everybody. And the options for programmable logic parts are even worse. There’s very scarce availability of any programmable logic parts, and the few that are available are actually less capable than the old Xilinx part, or cost five times as much. Sure, you can sometimes find parts with a few hundred at one supplier and 1000 at another, but it’s usually one specific oddball package or variant while the rest of the family is out of stock. That’s a fluke, not a reliable supply. To feel comfortable betting the farm on a new chip, I’d want to see all the members of its family with deep stock in the many thousands, at multiple suppliers.

If I redesign the Floppy Emu hardware, should I redesign around the best available parts options today, or around a blue-sky vision of the best-suited parts for the task regardless of current availability? A redesign will be a major time-consuming effort, and I can’t afford to get it wrong. It’s a gamble. If I redesign around the best available parts today, that greatly limits my choices. I could be locked into sub-optimal parts for a long time to come. That might make it much harder to add new features, or might create new supply problems in the future, or increase my costs. If I redesign around the parts I’d like to use in a perfect world, then I could have to wait a very long time before they become available again. The redesigned hardware might never see the light of day except for a few prototype units.

Given how much time any redesign would require, and the uncertain results, there’s another option I’m seriously considering: do nothing. Place some back-orders now for the ATMEGA1284 and XC9572XL, and then forget about it. Use the coming months to work on Yellowstone features or develop some cool new product, instead of redesigning the Floppy Emu. Hope that the parts become available before I need them, and if they don’t, it will be lean times at BMOW. I would still probably want to revisit the Floppy Emu hardware design in another year or two, to take advantage of the capabilities of newer parts, but I would do it after the chip shortage is over and there’s a clearer picture of what parts I can safely bet on.

Read 14 comments and join the conversation 

Amazon Pay, Part 2: $2300 Lost and Found

A few weeks back I wrote about my struggles with Amazon Pay, and $2300 of customer payments I was unable to move to my bank because of a mysterious recurring transfer failure. Amazon Pay’s customer support was initially so robotic and ineffective that I gave up hope for ever finding a resolution, or even finding someone who could understand what I was reporting. My first angry Amazon Pay essay attracted the attention of someone higher up at the company, who apologized and was very attentive to my case, but wasn’t able to do much to explain why all the bank transfers were failing. Now after seven weeks of trying, I’m happy to report that the long-expected money is finally in my bank account. Who identified the problem and fixed it – Amazon Pay support? My bank? Nope, it was me.

Here’s what happened: my business bank account number is 10 digits, but the attempted disbursement transfers from Amazon Pay were consistently coming in with only 9 digits in the ACH transfer data, with the last digit missing. This caused several days of delay for each attempted transfer, culminating in eventual failure and reversal of the funds. I confirmed with my bank that the 10 digit account number was the correct one to use, and this number works just fine for ACH transfers with several other business services that I use.

But… I also have a personal account at this same bank, and it has a 12 digit account number where the first two digits are zeroes. I asked the bank about this, and they said the zeroes aren’t required on their end, but it’s OK if they’re present. So I went back to Amazon Pay, re-entered my account number as a 12 digit number with two leading zeroes, and boom! Money delivered.

Why didn’t the regular 10 digit account number work here? I think the only possible explanation is a serious bug in Amazon’s payment systems. That seems tough to believe, given how many transfers a company like Amazon must process every day, and the size of their team dedicated to electronic banking. But I’m confident I didn’t mistype the account number at Amazon Pay the same way multiple times, and the regular 10 digit number works fine with other payment processing services, my payroll processor, the California tax authority, and electronic transfers to suppliers.

Maybe Amazon Pay doesn’t allow for the possibility that a single bank might have different types of accounts with different numbers of digits? I just don’t know.

Whatever the underlying cause, I can understand that bugs exist. My real gripe is the Amazon Pay customer service experience, which was initially so unhelpful that I couldn’t get anyone to even understand what I was describing. Over and over, I was told the same thing about how to set up my bank account, with agents reading from the same script. I explained that I’d already set up the bank info, and then the attempted disbursement transfer had failed, resulting in the automatic removal of the bank info, multiple times. But this didn’t seem to be a node in the customer support decision tree. And rather than getting a reply like “that doesn’t sound normal and I’m referring this case to a banking specialist”, I just got scripted replies for problems that were not my problem, while agents seemed to willfully ignore the specific details I was telling them. To make matters worse, my support case was twice closed by Amazon even though it wasn’t resolved, with no way to re-open it, so I had to open new support cases and start the whole song and dance over again.

After my first essay about this mess, I was contacted by someone higher up at Amazon Pay. He apologized for my bad experience, and said he would talk with the customer support team and review their procedures to help prevent something like this from happening again. He was very nice, and I appreciated his help. But if I hadn’t possessed enough minor internet fame to get decent attention on that first essay, would I ever have heard from this guy? Or would I have been stuck in tech support purgatory forever? What’s most concerning is that even after this new person intervened, he still wasn’t able to solve the problem or find any explanation for it. In the end, it was up to me to hit on the solution. The Amazon rep promised to share the details with the appropriate engineering team, so maybe the next customer of my bank who signs up for Amazon Pay won’t have the same trouble that I did.

Read 5 comments and join the conversation 

Hello New Zealand, Goodbye Hong Kong, International Shipping Dumpster Fire

International shipping in recent months has been a total disaster. Anyone who hopes we’re getting back to normal on this global supply chain stuff will be very disappointed. Between COVID-19 emergencies, a shortage of commercial airline flights to carry mail, customs clearance backlogs, and local destination transit service and postal system restrictions, most international shippers are experiencing major service disruptions. It’s especially bad for shipments sent by USPS First Class Package International Service, the cheapest available option and the one that most BMOW customers choose. FCPIS disruptions can sometimes cause unexpected delays of weeks or even months, and FCPIS shipments to some countries have been halted entirely. Pretty much every day I hear from a customer outside the USA asking “why is my shipment taking so very long?” Just how bad is it? A customer in Spain recently told me of an FCPIS shipment that had just arrived after four months. Ouch.

Since this post is specifically addressing people outside the USA, I should probably clarify that “dumpster fire” is American slang for a chaotic or disastrously mishandled situation. Because when things get really bad, we Americans apparently like to run around searching for open dumpsters full of garbage, then set their contents aflame while rubbing our hands with glee at the unholy mess. Which is a pretty good metaphor for international shipping today.

The US Postal Service “temporarily” suspended most mail delivery to Australia on September 3, and it remains suspended more than four months later. New Zealand service was similarly suspended on October 1. The lone bright spot here is the resumption of some classes of mail service to New Zealand on January 14. But on the same day, USPS mail service to Hong Kong was halted. Win some, lose some. Sorry Hong Kong.

 
Insanity is doing the same thing over and over and expecting different results

This is all becoming a familiar refrain. A search of the BMOW blog turns up many past laments about the difficulties of international shipping:

2013 International Shipping Hurts!
2014 International Shipping Meltdown
2016 Crikey! USPS International Shipping Costs
2018 International Shipping Struggles
2020 Sorry, Europe. International Shipping Woes
2020 The International Shipping Mess
2021 US Customs Export Control Says: I’m Screwed
2021 Temporary Suspension of Australia Shipping
2021 New Zealand Shipping Suspension

 
Something Better?

In an attempt to chart a path around this mess, BMOW has started offering some new shipping options. For most countries you’ll see an “Economy International” shipping option, which is priced similarly to FCPIS, and with a similar median delivery speed, but (hopefully) better worst-case delivery speed. It uses a crazy three-legged delivery scheme in which BMOW mails packages to a sorting warehouse in the USA, then a private shipping company applies a new shipping label with local postage for the destination country. Shipments going to the same country are batched together and sent in bulk by private shipper. Once the bulk shipment reaches the destination country, the individual parcels are dropped into the nearest mailbox for final delivery. If you’re wondering how this complex process could possibly be faster than simply mailing a package directly to the destination, let me refer you back to the FCPIS dumpster fire above.

For those customers who are willing to pay a little more for shipping, BMOW now also offers DHL and UPS shipping to most countries. This arrives in about a week, and is generally much more reliable than FCPIS, with better and more-detailed package tracking too. But it comes at a price: around 40 to 45 dollars, or more than twice the cost of typical FCPIS shipping. This might be acceptable for a customer who’s buying 250 dollars worth of hardware, but few people are willing to pay 40 dollars shipping on a 40 dollar purchase.

I’ll continue to hope that the international shipping landscape begins a return to normalcy later this year. Until then, thank you for your understanding as BMOW copes with this difficult shipping environment.

Read 1 comment and join the conversation 

Yellowstone Manufacturing is Go!

The i’s are dotted, the t’s are crossed, and Yellowstone manufacturing is underway! I want to say something satirical like “I never thought I’d live to see this day”, but truthfully this has been a long, long, loooooooong time in the making. General availability of Yellowstone is expected sometime around the middle of March. Development started in the summer of 2017, so you can do the math on total development time.

I’ve created a Yellowstone product page. This will be the official home for everything Yellowstone, including an overview of what it is and why you might want it, compatibility information, links to the instruction manual, firmware updates, and more.

Yellowstone is a universal disk controller card for Apple II computers. It supports nearly every type of Apple disk drive ever made, including standard 3.5 inch drives, 5.25 inch drives, smart drives like the Unidisk 3.5 and the BMOW Floppy Emu’s smartport hard disk, and even Macintosh 3.5 inch drives. Yellowstone combines the power of an Apple 3.5 Disk Controller Card, a standard 5.25 inch (Disk II) controller card, the Apple Liron controller, and more, all in a single card.

Need to attach a disk drive to your Apple II? Yellowstone should be your first choice, because it does virtually everything that every other Apple disk controller can do, plus more. The retail price for Yellowstone is planned somewhere in the mid-$100s range. This is a nice value, given that an Apple 3.5 Disk Controller Card costs $200+ and used Liron cards sell for $300+ on eBay, and Yellowstone can do much more than either of those cards.

 
Apple Disk Controller Card Comparison

Disk Controller Supports 3.5 inch drives Supports 5.25 inch drives Supports smart hard drives Supports Macintosh drives 20-pin ribbon connector DB-19 connector Number of connectors
Disk II
Controller Card
        2
Disk 5.25
Controller Card
        1
Apple 3.5 Disk Controller Card       1
Apple Liron Card         1
Yellowstone 1 2

[1] optional DB-19F connector

Since I know people are going to ask, I’m not taking pre-orders at this time, but you can sign up for the BMOW Newsletter if you want to be informed when Yellowstone is ready for sale. There should be plenty of Yellowstone cards to meet demand for at least a few months, so there’s no worry of an immediate sell-out. But in the medium to long term, the global semiconductor shortage and general meltdown of supply chains will be a problem. I couldn’t build additional Yellowstone cards right now, even if I wanted to, because the necessary parts just aren’t available anywhere. So if you want one of these, maybe don’t wait too long beyond March to grab one.

Want all the nitty-gritty usage details? You can read the Yellowstone instruction manual here. Now the waiting begins…

Read 23 comments and join the conversation 

Older Posts »