International shipping in recent months has been a total disaster. Anyone who hopes we’re getting back to normal on this global supply chain stuff will be very disappointed. Between COVID-19 emergencies, a shortage of commercial airline flights to carry mail, customs clearance backlogs, and local destination transit service and postal system restrictions, most international shippers are experiencing major service disruptions. It’s especially bad for shipments sent by USPS First Class Package International Service, the cheapest available option and the one that most BMOW customers choose. FCPIS disruptions can sometimes cause unexpected delays of weeks or even months, and FCPIS shipments to some countries have been halted entirely. Pretty much every day I hear from a customer outside the USA asking “why is my shipment taking so very long?” Just how bad is it? A customer in Spain recently told me of an FCPIS shipment that had just arrived after four months. Ouch.

Since this post is specifically addressing people outside the USA, I should probably clarify that “dumpster fire” is American slang for a chaotic or disastrously mishandled situation. Because when things get really bad, we Americans apparently like to run around searching for open dumpsters full of garbage, then set their contents aflame while rubbing our hands with glee at the unholy mess. Which is a pretty good metaphor for international shipping today.

The US Postal Service “temporarily” suspended most mail delivery to Australia on September 3, and it remains suspended more than four months later. New Zealand service was similarly suspended on October 1. The lone bright spot here is the resumption of some classes of mail service to New Zealand on January 14. But on the same day, USPS mail service to Hong Kong was halted. Win some, lose some. Sorry Hong Kong.

 
Insanity is doing the same thing over and over and expecting different results

This is all becoming a familiar refrain. A search of the BMOW blog turns up many past laments about the difficulties of international shipping:

2013 International Shipping Hurts!
2014 International Shipping Meltdown
2016 Crikey! USPS International Shipping Costs
2018 International Shipping Struggles
2020 Sorry, Europe. International Shipping Woes
2020 The International Shipping Mess
2021 US Customs Export Control Says: I’m Screwed
2021 Temporary Suspension of Australia Shipping
2021 New Zealand Shipping Suspension

 
Something Better?

In an attempt to chart a path around this mess, BMOW has started offering some new shipping options. For most countries you’ll see an “Economy International” shipping option, which is priced similarly to FCPIS, and with a similar median delivery speed, but (hopefully) better worst-case delivery speed. It uses a crazy three-legged delivery scheme in which BMOW mails packages to a sorting warehouse in the USA, then a private shipping company applies a new shipping label with local postage for the destination country. Shipments going to the same country are batched together and sent in bulk by private shipper. Once the bulk shipment reaches the destination country, the individual parcels are dropped into the nearest mailbox for final delivery. If you’re wondering how this complex process could possibly be faster than simply mailing a package directly to the destination, let me refer you back to the FCPIS dumpster fire above.

For those customers who are willing to pay a little more for shipping, BMOW now also offers DHL and UPS shipping to most countries. This arrives in about a week, and is generally much more reliable than FCPIS, with better and more-detailed package tracking too. But it comes at a price: around 40 to 45 dollars, or more than twice the cost of typical FCPIS shipping. This might be acceptable for a customer who’s buying 250 dollars worth of hardware, but few people are willing to pay 40 dollars shipping on a 40 dollar purchase.

I’ll continue to hope that the international shipping landscape begins a return to normalcy later this year. Until then, thank you for your understanding as BMOW copes with this difficult shipping environment.

The i’s are dotted, the t’s are crossed, and Yellowstone manufacturing is underway! I want to say something satirical like “I never thought I’d live to see this day”, but truthfully this has been a long, long, loooooooong time in the making. General availability of Yellowstone is expected sometime around the middle of March. Development started in the summer of 2017, so you can do the math on total development time.

I’ve created a Yellowstone product page. This will be the official home for everything Yellowstone, including an overview of what it is and why you might want it, compatibility information, links to the instruction manual, firmware updates, and more.

Yellowstone is a universal disk controller card for Apple II computers. It supports nearly every type of Apple disk drive ever made, including standard 3.5 inch drives, 5.25 inch drives, smart drives like the Unidisk 3.5 and the BMOW Floppy Emu’s smartport hard disk, and even Macintosh 3.5 inch drives. Yellowstone combines the power of an Apple 3.5 Disk Controller Card, a standard 5.25 inch (Disk II) controller card, the Apple Liron controller, and more, all in a single card.

Need to attach a disk drive to your Apple II? Yellowstone should be your first choice, because it does virtually everything that every other Apple disk controller can do, plus more. The retail price for Yellowstone is planned somewhere in the mid-$100s range. This is a nice value, given that an Apple 3.5 Disk Controller Card costs $200+ and used Liron cards sell for $300+ on eBay, and Yellowstone can do much more than either of those cards.

 
Apple Disk Controller Card Comparison

[1] optional DB-19F connector

Since I know people are going to ask, I’m not taking pre-orders at this time, but you can sign up for the BMOW Newsletter if you want to be informed when Yellowstone is ready for sale. There should be plenty of Yellowstone cards to meet demand for at least a few months, so there’s no worry of an immediate sell-out. But in the medium to long term, the global semiconductor shortage and general meltdown of supply chains will be a problem. I couldn’t build additional Yellowstone cards right now, even if I wanted to, because the necessary parts just aren’t available anywhere. So if you want one of these, maybe don’t wait too long beyond March to grab one.

Want all the nitty-gritty usage details? You can read the Yellowstone instruction manual here. Now the waiting begins…

This is a story of a small e-commerce merchant who signed up for Amazon Pay: a payment processing service similar to PayPal that leverages customers’ existing Amazon accounts. It all seemed so promising. Amazon Pay would provide an easier checkout experience for my customers, with the shipping address and payment details already filled, and I’d save a small amount in transaction fees. What could go wrong? But six weeks later, Amazon has $2300 of my money with no way for me to access it, and their support has been maddeningly worthless. I can only hope that a public shaming will spur somebody at Amazon to take notice and intervene, otherwise that money may just be gone.

I created the Amazon Pay account on November 29, and configured my business bank account info so I could receive electronic disbursements. This is a normal business checking account at a normal US bank, using US dollars, and that works fine with other payment processors like PayPal and Shopify Payments. I entered the bank’s ACH routing number and my account number, which ends with the digits 622. Soon I was able to receive payments in my store using Amazon Pay, and everything seemed good.

The problem started on December 8, when Amazon Pay attempted the first disbursement to my bank account. I received a failure notification by email:

“Your most recent transfer of funds in the amount of $XXX was not successfully deposited into your bank account on file (ending in 622). The funds were returned to us, and we have removed the current bank account information from your account.”

Amazon Pay’s merchant support was unable to explain what went wrong. They sent me some generic troubleshooting info. It’s not clear why they couldn’t provide any specific details about why this transfer failed. The generic info suggested the routing or bank account numbers might be wrong, so I re-entered the numbers again, being extra careful to get them correct. Then I waited a week… and the exact same thing happened. This same dance played out five more times on December 16, 17, 18, 20, and 21. Each time Amazon Pay attempted a disbursement to my bank, then a few days later I got the same email saying the money wasn’t successfully deposited to my bank account ending in 622, and “we have removed the current bank account information from your account.”

On December 23 I contacted my bank and gave them the trace IDs for all the failed transfers. My bank determined that the account number was wrong on every transfer. All the failed ACH transfers used an account number ending in 62 instead of 622: the last digit of the account number was missing!

Armed with this information, I returned to Amazon Pay merchant support, and here’s where I really started to get frustrated. I discovered that my original support case had been unilaterally closed without resolution, because five days had passed since the last reply, and apparently that’s their support policy. It’s not possible to reopen or reply to a closed case, so I had to create a new case and explain the issue all over again. This did not go well. Imagine this support exchange happening slowly by email over a series of two days:

Me (paraphrasing): I’m still having trouble with disbursements. You closed my last case number XXXX about this issue, so I had to create this new case. My bank says the last digit of the bank account number is missing on all the ACH transfers from Amazon Pay, but the bank account number is correct in my Amazon Pay settings.

Amazon Pay: To receive disbursements, you need to set up your bank account info in your Amazon Pay settings. You do not have a bank account configured in your settings.

Me: Please see attached screenshot showing my bank account info is already set up correctly. I have already done this several times, but every time there’s a failed disbursement attempt, Amazon Pay removes my bank account information. The fundamental problem is that the last digit of the bank account number is missing in all of Amazon’s ACH transfers. Please look at the ACH transfers yourself to confirm.

Amazon Pay: To receive disbursements, you need to set up your bank account info in your Amazon Pay settings. You do not have a bank account configured in your settings. The screenshot you sent doesn’t show what account it’s associated with so we can’t tell anything from that.

Me: The bank account info has already been entered. I’ve attached a full-page screenshot showing this. The deposit method isn’t assigned, because every time there’s a failed disbursement attempt, Amazon Pay removes my bank account information. Please respond to the fundamental issue of the missing digit in the ACH transfers. Here are all the trace IDs again. You can verify them yourself.

Amazon Pay: Your bank account may not be compatible with our system. Your account needs to be ACH-ready. Please contact your bank.

Me: Yes it’s ACH-ready, it’s a standard business checking account at a USA bank, and I’m already receiving ACH transfers there from PayPal and Shopify. I mentioned all this already in the first case that I opened, that Amazon Pay closed without resolution. You still have not addressed or even mentioned the fundamental problem that I’ve asked about four times now, the missing digit in the ACH transfers. Kindly investigate why this digit is missing in all the ACH transfers from Amazon Pay.

Amazon Pay: I understand that you are having problems with your bank account. Thank you for your continued patience. At this time I am working with our internal team to reach a resolution on your case.

(five days later) Me: Has there been any progress on this? I’m still unable to access the funds in my Amazon Pay account.

Amazon Pay: I understand that you are having problems with your bank account. Thank you for your continued patience. At this time I am working with our internal team to reach a resolution on your case.

(five days later): Case closed with no further response from Amazon. Can’t reopen or reply.

Now I was really upset. Since I couldn’t reopen the old cases, I opened a third case to explain the issue with failed disbursements and the missing account number digit, all over again.

Amazon Pay: To receive disbursements, you need to set up your bank account info in your Amazon Pay settings. You do not have a bank account configured in your settings.

Me: Oh my God not again.

I was able to reach a phone support agent, and told him I just wanted to close my Amazon Pay account and get a final payout by paper check, since they seem unable to resolve the problem with electronic transfers to my bank. I was told they can’t do it. Basically the options were 1) electronic transfer to bank, or 2) electronic transfer to bank.

I asked the phone agent to review the previous case, and see what information the “internal team” had found before that case was closed. There was none. All the discussion from the previous case was essentially gone and worthless.

So I went through all the same conversation again with the phone agent, arguing about why my bank account info wasn’t set up in my Amazon Pay settings. I gamely played along while he had me delete my browser cookies and other irrelevant troubleshooting. Since closing the account didn’t appear to be a viable option, I tried once again to delete and re-enter all my bank account info. The phone agent insisted that everything would be OK now that I’d entered my bank account info, and I would receive a disbursement within the next few days. He didn’t seem to understand or care that I’d already been through this same loop several times before without success. Skeptical, I thanked him and hung up. I then received a follow-up email:

Amazon Pay: I understand that you would like to know how you can see the transaction and how disbursements work for the Amazon Pay platform. If you want to find your transactions coming from Amazon Pay just remember to switch from “Amazon.com” to “Amazon Pay (Production View)”. You would be able to see that next to your Seller name and the USA flag where there is a scroll down. etc…

What the actual fuck?! What is this and how is it relevant in any way? Is Amazon Pay’s merchant support even staffed by human beings, or is it all just chatbots and GPT-3-driven natural language text generation? Amazon needs to understand that merchants have many choices for payment processing, including PayPal and traditional credit card processors, who have professional support staffs to respond to account problems in a timely and constructive way. Many people complain about PayPal, but at least PayPal’s support team responds to me with useful and relevant information that answers my questions. After a month of trying, I’ve failed to get anyone at Amazon Pay to even acknowledge my problem with the missing account number digit, let alone give me any constructive help. Their merchant support seems designed to frustrate merchants into simply giving up and going away. I’ve lost $2300 thanks to Amazon Pay, with no further options I can see for resolution. Meanwhile Amazon Pay has been removed from my store, and I will not be reactivating it. Good riddance.

As 2021 draws to a close, Yellowstone is very nearly finished, with just a few details left to iron out. I recently tested a new v2.3 prototype with a couple of small PCB changes, and so far it’s looking good. The new PCB fixes an obscure problem with the Unidisk 5.25 and Disk IIc, where one of those specific drives connected as Drive 2 would interfere with another 5.25 inch drive at Drive 1. Initially I’d planned to write this off as a known compatibility issue, but I had second thoughts, and rushed through a new PCB revision.

Parts sourcing continues to be a major problem. The good news is that I was able to find more of the FPGAs that Yellowstone needs, but I had to go through a third-party dealer in Hong Kong and pay roughly twice the normal price. I have enough parts now to manufacture about 500 Yellowstone cards. Beyond that, the outlook is murky. I expect the semiconductor shortage to get worse before it starts getting better, and many parts are currently quoting lead times of a year or more. Given the current environment, Yellowstone should probably be considered a “limited edition” in 2022, with the possibility of some restock in 2023 or 2024.

I’ve received a couple of manufacturing quotes already from CircuitHub and MacroFab, two vendors with automated web-based quoting tools. These are helpful sanity checks and estimates, but they’re not yet fully baked quotes that I could move ahead with as-is. Optional features like beveled PCB edges and gold fingers require a custom quote. Programming and testing requirements are also difficult to factor in to automated quoting tools.

The preliminary information looks like beveled PCB edges and gold fingers would add about 30 percent (combined) to the total cost. I don’t know the cost breakdown between the two features. 30 percent is quite a lot, and I’m debating whether people will be willing to pay 30 percent more for a card with these features. The prototypes have square edges and ENIG fingers, and it doesn’t seem to have been an issue for the beta testers, so I’m leaning against including these extra features.

I talked to the support staff at Circuit Hub, and they were reluctant to discuss using my own-designed Yellowstone tester. They felt that the quality of their assembly process was so high, with automated visual and x-ray testing, that additional functional testing was unnecessary. And they stressed that if an assembly problem were ever found, I could send the board back to them for free rework. I admit to lacking experience here, but this makes me uneasy. It puts the onus on me to actually test each board before it’s sold, instead of having the vendor do it, but that’s exactly the type of work I want to outsource. And it assumes that free rework would make it acceptable to receive faulty boards. The cost of faulty boards is mostly lost time for testing and troubleshooting, not the cost of rework. By the time I’ve tested a board myself, confirmed that it fails, diagnosed the problem, and identified it as an assembly problem, I could probably just fix it in five minutes with a soldering iron. I’m not going to ship a board back for that. But I don’t want to be doing that kind of work in the first place. To be fair, they did say they would do functional testing if I really wanted it, but their process doesn’t seem to be designed for this, and they implied that it would be expensive.

I have the general impression that the automated quoting vendors like CircuitHub and MacroFab are geared more towards low volume prototyping, or simple projects involving common parts. I think they may not have the most competitive prices either, but we’ll see. For anything slightly non-standard, or for the most competitive pricing, I think it may still be necessary to go the traditional route of phoning or emailing a vendor, and having an actual person-to-person discussion about manufacturing options and requirements. I’ll be doing that soon.

One other interesting option is Seeed Studio’s Fusion PCB service. They have a semi-automated quoting tool, but the tool still needed some manual help to finish my quote, and even then it wasn’t entirely correct. However, the total quoted cost was less than half the cost from the two other vendors. They even agreed to include functional testing for free, but other options like beveled edges and gold fingers don’t seem to be available at all. The big downside is the two-way trans-Pacific shipping that would be needed. I would need to ship all the FPGAs to them for assembly, which they don’t really like doing, and which would create extra paperwork hassles and tax/tariff concerns. I would still prefer to work with a local vendor, or at least a vendor in the same country, if I can find one that’s competitive.

Most vendors are quoting lead times of about two months, so if I can get started with manufacturing soon, I could have product ready for sale by March. The lunar New Year holiday is fast approaching though, and that usually shuts down most Chinese businesses for a couple of weeks. That may delay the schedule, depending on what vendor and manufacturing process I ultimately choose. But barring any further unexpected problems, final Yellowstone cards should be available by March or April at the latest. Woohoo!

Yellowstone is inching slowly towards the start of manufacturing. One question that’s arisen is the type of surface plating to use on the PCB. The default / cheap plating is HASL or Hot Air Solder Leveling, which is just a thin layer of solder, consisting mostly of tin. This is what BMOW’s other products use. But engineering wisdom says that for an edge connector, or any PCB surface that will send electrical signals across a mechanical contact surface, you should use ENIG plating. ENIG is Electroless Nickel Immersion Gold, and it’s a layer of nickel covered in a second layer of gold. It’s more expensive, but more durable.

So ENIG then? Well, maybe not. I recently learned about a problem called galvanic corrosion that occurs when two dissimilar metals are in contact for a long time. The peripheral card slots in an Apple II computer have tin fingers, I believe – at least they’re not obviously gold-colored. Does that mean an ENIG plated board inserted into an edge connector slot with tin fingers is doomed to contact corrosion and premature failure? A quick check of other Apple II peripheral cards in my office showed they all have gold-colored connectors. I’m uncertain if they’re ENIG or something else, but probably ENIG.