During the weekend WishList Products experienced an unexpected situation that effected roughly 10% of our customers (which has since been resolved).
We’d like to inform you of what happened along with what we have done to correct it and prevent it from happening again.
When WishList Member is purchased, each customer is issued a “license key” to verify the purchase.
With that in mind, WishList Member checks the license key on an ongoing basis to verify that each site is using the correct license and to also issue product updates to these sites accordingly.
The problem that occured this weekend resulted because the “activation server” that verifies the license key, went down.
So for the sites where WishList Member wanted to verify the license, there wasn’t anything there to verify it was a valid license (because our server was down).
This caused roughly 10% of our customers to receive a “500 internal error”.
As soon as we heard about our server being down (we heard first from our customers vs. our hosting provider), we immediately sprung into action to resolve the issue.
This being the first time something like this has ever happened in our company’s history (3+ years), we weren’t sure what was all of a sudden causing this server to crash.
Nevertheless, we got the server back up, everything was fixed and after monitoring it for several hours, it looked to be running smoothly.
Then, the following day it happened again.
This was baffling to us as this was two times in as many days and prior to that, it had never occurred.
Why was this all of a sudden happening?
We immediately got the server back up and then went into “investigation mode”.
After hours of analyzing the situation, we found out that it had to do with specific limitations of the server we were using (something we didn’t initially foresee).
That’s when we decided to make some significant changes in the way our verification process operates.
We have now implemented a situation whereby the “verification check” must fail multiple times before the WishList Member license is deactivated.
Plus, even when it is going to be deactivated, the customer will have a number of days prior to the deactivation and will be made aware of that with several messages.
This will eliminate the situation we had this weekend because the sites would continue working fine until the next check. If the next check failed as well, the customer would still have plenty of time to prepare as well.
The time for this whole process to complete before the final deactivation would be roughly a couple of weeks. So, as you can see, this changes the situation completely from what happened this weekend when our server was down for a few hours.
Furthermore, we have made significant upgrades to our server environment introducing load balancers, mirroring servers (so if one goes down, the other kicks in to verify licenses) etc.
We are deeply sorry to those customers who were effected.
We take this very seriously as we pride ourselves on our products and the service we provide to our customers.
In addition, we too have sites using our own software (and we were effected as well – and we don’t want anything like that happening either).
Moving forward, we are very confident in the changes we have made and look forward to continuing to hear about your success.
Co-Founder, WishList Products