eCommerce News

IMPORTANT: problem may mean missed orders

By Josh O'Connell on January, 23 2019

An unannounced change to's gateway, released as a security update Tuesday, led to potential issues with hundreds of thousands of eCommerce stores. Among the platforms impacted were Zoey and Magento.

In this update we'll share with you what happened, how this impacts your store and steps you should take if you're an user on your Zoey store. Although the immediate problem of orders not properly processing is fixed, you will need to check to ensure that all payments captured on January 22, 2019 are tied to an order.

What happened

Zoey first received report of this issue from customers at 3 p.m. EST Tuesday, that they were getting errors when orders were being placed with as the gateway. In some cases a white screen was appearing. Orders were also not always showing up in Zoey, but were still being captured in As Zoey had not made any changes on its side, we began to research what could be causing the issue.

After speaking with at length, it was clear they had made a change earlier in the day as part of security updates. The change implemented a rule they had previously documented, but not enforced, when it came to the size of the delimiter being used when passing data back and forth. With the update, reduced the size of the allowed delimiter to one character.

Magento and Zoey, among other solutions, used a multi-character delimiter to limit risk of data entry by customers tripping the delimiter by accident. When changed the size, it effectively broke the ability for carts using multi-character delimiters to process orders consistently.

Delimiters are used to separate one piece of data from another - they're used to help systems understand when one chunk of data ends and another begins. Changing what delimiters are supported on one system requires the other system to be updated as well, to ensure that both sides can interpret the data coming from the other. In this case, an unannounced change on one side forced an urgent, unplanned change on the other.

How this could happen

Normally, such a change is publicized well in advance, with the ability for eCommerce solutions to make updates prior to the change. It's uncertain why this did not happen in this case, as it put at risk a large amount of stores. Magento 1 is still a heavily used cart, despite its announced end of life, and when patches have to be applied, each store must be done separately. Zoey's SaaS architecture allowed us to patch all stores quickly; within a few hours of the original report, we had a patch in place for all customers to fix the immediate problem, with all stores updated by 7:30 EST.

While announced other changes that impacted their security updates, as of this posting, it does not appear that the delimiter change was publicized during testing of the release.

Resolving the immediate issue representatives have told us they are rolling back the release for now. However, they could re-release the change, so it's important that those impacted by it correct the issue. As part of Zoey's efforts to assist the community, Zoey President Uri Foox shared a temporary solution based on Zoey's fix to the Magento community.

Zoey's fix will remain in place despite the rollback, with the expectation that in the future they will revive the new rules. This will ensure that we don't have a repeated problem on Zoey.

How this impacted your store

If you use as your gateway, two problems occurred, one of which is resolved, while the other requires your attention now.

  • For a number of hours Tuesday, some orders failed because of's change. This issue has been resolved, per the above.
  • Because of the issue, some charges were processed without an order being completed by Zoey. This is because got the data and was able to charge cards, but was not able to return accurately the successful processing of the charge, instead passing back an error that led to scenarios such as white screens. In some cases, customers would attempt their order multiple times, and so multiple charges would be made and only one, or sometimes even no, orders would show up in Zoey.

For the latter issue, we are encouraging all customers to audit their transactions against their Zoey orders. If you find charges that aren't tied to an order, we recommend reversing them and contacting the customer to let them know they will need to re-submit their order. Merchants can search transactions by business day (Tuesday, January 22 is the only affected day, and only through 7:30 PM ET), click on a transaction, and find an email address attached to a charge to get contact info for that charge. You can log in to your account at this link.

In a few cases, due to a temporary fix being implemented while we worked on a permanent one, There may be a mis-match of information between and the order in Zoey, meaning credit memos and other actions cannot be done in Zoey and automatically be tied back to In those cases manual processing will need to be done in, and then Zoey updated to be in lockstep.

Zoey updated merchants via its status page as the research and fix was taking place. As a reminder, we encourage customers and partners to visit the Status page and subscribe to alerts to get emails when issues occur.

Additional reading

A number of community posts have gone up since this issue was discovered. Many of these posts are more technical in nature, but provide additional detail as to what happened.

Recommended Articles

Submit a Comment

Subscribe and stay up to date