In the wake of Crystal Web launching head-on into financial ruin and EvoNet’s management of the brand causing confusion for everyone, Crystal Web’s customers who took out ADSL or fibre bundles with the company still find themselves without internet more than two months later. Having gone through the motions of trying to recover payment from the company, Openserve suspended the company’s access to internet services that they managed for their customers, and EvoNet’s brand new customers found themselves unable to get support for their leased lines and unable to get answers as to what happened. Openserve last month opted to take charge of the situation in a bid to do right by Crystal Web’s customers, and began the process of moving all affected lines into the holding pool to allow customers to get another ISP to manage their line.

NOTE: For the purposes of this article, Crystal Web is referenced differently when talking about the company and the Crystal Web brand owned by Paul Hjul and leased to EvoNet as part of an agreement over the purchase of the company’s customer book. Crystal Web the brand will always appear italicised. Questions regarding the line suspensions and the company’s plans to rectify the situation went unanswered by EvoNet by the time of publication. For clarification, “access services” is the correct terminology when talking about a phone line or fibre line’s ability to connect to the internet, as the access service is added on to the physical line as a service.

After the previous article, I contacted Openserve and asked about reports that were coming in that Crystal Web customers whose lines had been suspended were being unexpectedly moved into the holding pool. One of the biggest pieces in the Crystal Web saga is that the company lost control over the DSL and fibre access services they managed for their customers when their service with Openserve was suspended. Chatting to the EvoNet support staff proved to be pointless because they had no control over the services. As of 21 November, Openserve opted to move all of the lines that had a suspended DSL or fibre service back into the holding pool so that customers wouldn’t be adversely affected.

“We had already exhausted all the processes to recover payment from the service provider, Crystal Web,” said Openserve’s Pynee Chetty, Media Relations specialist. “We later became aware that customers were taking the brunt of the situation.”

In my conversation with Chetty I learned of some of the pains Openserve had gone through with Crystal Web. When I wrote that Openserve’s only involvement in the events was that they weren’t being paid, that wasn’t strictly true. Openserve had been trying to recover payment from Crystal Web management for months. Crystal Web had been asked multiple times to keep up with payments, and Openserve had used all their regular processes to do this (like letters of demand, etc.). I got the impression that Openserve was met with zero communication from the people that should have been in charge of fixing the problem and paying their service provider.

Chetty also explained that the situation was so unusual that Openserve didn’t have a standard way of dealing with it. In the normal course of business, ISPs are the ones to release or manage a line’s access services at a specific address, and take it out of, or put it back into, the holding pool, so that an ISP may manage it on behalf the customer. But when Crystal Web‘s customers couldn’t process their services, and when Crystal Web took upwards of six weeks to process cancellations and moving lines into the holding pool, no standard policy that could apply. An ISP ignoring both their customers and their service provider was unthinkable.

Openserve is also a wholesale provider to ISPs and not the general public. By design they have no consumer-facing call centre or point of contact aside from the online form used to register your interest in a fibre line. Part of this is regulatory constraints, and part of it is because the responsibility to manage the access service on a copper or fibre line has been handed over to ISPs, who have various ways of interfacing with Openserve’s network through software APIs. My initial advice to call Telkom Direct retail services to fix the problem was well-intentioned, but incorrect – only the customer’s chosen ISP can interface with Openserve.

This was the funny side of the situation that stuck out to me. If you had a copper line with Openserve, and you subscribed to a voice package with Telkom and your DSL rental was handled by another ISP, calling Telkom was a good idea only if you wanted to move your line’s access services to Telkom Internet. If you had Openserve fibre, calling Telkom Direct to fix the problem would only work if you wanted to move the access service to them. Telkom Direct and Telkom Internet were in the same boat as other ISPs in terms of what they could do about the situation.

Still offline? Here’s what you gotta do

Thanks to a second and third wave of suspensions being reported on the MyBroadband forums, there’s a lot of misinformation out there as to how this is supposed to work. I have an outline of what you can do as an affected customer, but there are bound to be anomalies where things aren’t working out as expected. We’ll deal with those later. Here’s the gist of it:

  1. If you’re a current or ex-Crystal Web customer, and your line is currently syncing but not connecting to the net, you’re in the holding pool. You need to contact a new ISP, like Telkom Internet, or Afrihost, to claim your line from the holding pool to give you internet connectivity.
  2. If your line isn’t syncing, it may be suspended. You need to contact your new ISP to get them to liaise with Openserve to lift the suspension and move the line back into the holding pool.
  3. If your line is not suspended, and not syncing, and no-one knows what its status is, that’s a special case. Your new ISP needs to work with Openserve to figure it out.
  4. If your line is syncing and working, it’s an anomaly because this shouldn’t be possible, so expect a suspension eventually when the system catches up.

When the batch move into the holding pool was completed, Openserve went to contact customers who had been moved to tell them that they had 30 days to find another ISP before their access service would be decommissioned. Openserve only had a partial database of contact details for people who had been moved into the holding pool. If anyone didn’t get an email, a phone call, or an SMS, you’d need to give your physical address to your ISP to get them to figure out what the status of the line is.

As for the possible litigation I reported on, where Openserve could be taking Crystal Web to court to recover the unpaid fees (which have now ballooned to R3 million-plus according to official documents that I have seen), Openserve’s Pynee Chetty stated that the company reserves its right with regards to its next steps in the matter with Crystal Web.

Openserve could not be drawn to comment on whether such a unique situation like this would cause the company to re-evaluate their policies and try identify gaps in their service where the unthinkable happens and an ISP’s customers are disproportionately affected. This probably will happen anyway. In the future the provider might be a little more aggressive in similar situations where payment isn’t being made by the ISP, and where they don’t wait as long as they did here to release all the customer’s lines in the holding pool, but that’s speculation on my part.

This raises the question of whether EvoNet is able to offer access services on DSL or fibre lines operated by Openserve or similar providers, whether a third party is managing them, or if they even have the necessary licences to offer internet connectivity on leased DSL and fibre lines in the first place. It turns out that they don’t, at least not currently, but more on that later.

ISPs might not be prepared for the fallout

When I first learned of the outstanding fees payable to Openserve, the figure was stated to be just over R2 million. A study conducted four months ago found that the average broadband speed in South Africa was just over 6.3Mbps (megabits per second). Thumbsucking a figure is required here, so if we take the average for Openserve’s DSL network to be around 4Mbps we come to a figure of approximately 6,600 subscribers that could have been affected by the line suspensions. That’s 6,600 people looking for a new ISP, and they’re all going to want to jump to someone else in the next few minutes if it were possible.

That’s not to say that all of Crystal Web’s leased line customers were on Openserve’s DSL network, but it’s going to be an overwhelming majority of them. Also, we’re discounting the number of people who may have just a data account through EvoNet, and plan to jump ship anyway. There are going to be a lot of people in that camp.

Are other ISPs prepared to deal with this highly unusual situation? The process of calling up Openserve to put the phone line’s internet service in their control is still the same, but the sales staff need to know that any anomaly for an ex-Crystal Web customer will mean manual intervention. There might even be errors that are spat out by the system that they don’t have training for, like the customer’s line being in limbo, seemingly not existing. There’s a high chance of additional frustration as customers are passed pillar to post with staff not giving them the correct information, especially with first-line sales staff that don’t want to go through the extra hassle to sort out the problem – we’ve all run into these people. This kind of situation may merit special training in the future to know how to deal with a customer’s request when there’s clearly something gone wrong with how the system might record something.

Meanwhile, back at the ranch…

Things are extremely quiet over at EvoNet. There hasn’t been a social media post on the Crystal Web Facebook or Twitter accounts since early October, and EvoNet’s official contact number on their website (when it isn’t redirecting customers to Crystal Web) hangs up almost immediately when you connect to it. Navigating to Crystal Web‘s site also still brings up the initial 8 October announcement to customers apologising for the service interruptions that suspended their lines. The lack of communication from EvoNet is deafening to frustrated customers who just want their internet working again. It’s understandable that companies aren’t always honest or transparent with their customers because it may inadvertently lead to litigation or investors taking their money elsewhere, but it’s never a good idea to stay completely silent.

It is ironic that Afrihost, a company dogged by their own past littered with the same kind of inaction that EvoNet is showing here, is seen on the MyBroadband forums offering to help affected customers get their lines under management by liasing with Openserve. It’s opportunistic, sure, but it’s laudable that the company’s support staff are seen going the extra mile.

Despite appearances that everything may be quiet, however, things behind the scenes are unsettling. I paste below a copy of my chat transcript with a Crystal Web support staffer that took place late on Friday, 13 November 2018, about two weeks ago.

Support Soldier: Hi Wesley
Support Soldier: how can i help you
Wesley: Hi there. I’m a journalist, and I’m writing a story about the line suspensions that’s been going on since last month. Is there any update to the dispute between CW and OpenServe yet?
Support Soldier: at this time Upper Management is working on the problem with Open Serve
Support Soldier: you are welcome to send a email to support@crystalweb.co.za for and update on the problem
Wesley: Alright. And is upper management Crystal Web, or EvoNet? I ask because it seems like EvoNet’s handling all the messaging, but CW picks up the brunt of customer complaints.
Support Soldier: Crystal Web Management is working on the problem, best is to jump on live chat in the morning and ask for one of the manager to reply to your questions

I mentioned in my earlier coverage that EvoNet’s handling of the Crystal Web brand, and the confusion they sow through not differentiating between the brand and the company, is causing anger and frustration for their customers. The Crystal Web brand is owned by Paul Hjul according to the agreement that EvoNet signed with then-current Crystal Web directors Shaun Kaplan, Karen Charlton, and Paul Hjul, but EvoNet gets to use the brand and the company’s social media for up to five years as if they were Crystal Web by extension.

When I contacted Support Soldier about the dispute, he was acting as support for the Crystal Web brand, but he put the onus on fixing the problem to “Crystal Web Management” and told me to visit the chat room again in the morning to talk to “the manager”. Ostensibly that’s someone who worked for Crystal Web, but that’s not the case because the company is now only one person, Paul Hjul. Whoever is acting as support staff for Crystal Web works for EvoNet, not Crystal Web. The messaging comes across as disingenuous.

The reason for me bringing this up again is because recent events muddle things up even further, and it’s becoming difficult to apply Occam’s Razor and assume that these are all honest mistakes.  I received a copy of the following chat with Crystal Web support in my inbox, which took place on 22 November 2018.

*** Support Soldier joined the chat ***
Chris: HI SS.
Chris: I was talking to Support General
Support Soldier: Hi Chrisjan
*** Support General joined the chat ***
Support Soldier: let me get him for you
Support General: Hi Chris
*** Support Soldier left the chat ***
Support General: Apologies for that long delay
Support General: The line belongs is manage by Crystal Web
Support General: we are unable to do anything on their behalf as Evonet
Chris: Now how do I solve this problem then?
Support General: Crystal Web’s contact will be Paul Hjul , he is the Main Director
Chris: He did not deem it necessary to reply to my e0mail.
Chris: He did acknowledge receipt.
Support General: They will have to resolve this for you
Support General: Because as Evonet we have no lines that we manage just as yet
Chris: How do I contact Paul Hjul?
Chris: complaints@crystalweb.co.za?
Support General: ******@crystalweb.co.za
Chris: THanks.

I have in my possession official communications from Openserve to Paul Hjul stating that Crystal Web’s ability to render access services on Openserve’s physical network was discontinued on 16 November 2018. From that point on, neither Hjul nor Crystal Web retain any capability to manage access services on active DSL or fibre services on Openserve’s infrastructure, which includes moving them into the holding pool. 16 November was a Friday. The batch move to the holding pool took place on Wednesday, 21 November.

EvoNet’s support staff may have been unaware of the batch move into the holding pool (which is unlikely given this post), or of the status of Crystal Web’s ability to manage lines on their client’s behalf (which they knew wasn’t possible due to the line suspensions), but this same story plays out in other chat transcripts that I’ve received. The irony that an EvoNet employee on the Crystal Web website is telling a Crystal Web customer to contact Crystal Web is unmissable. There is no hint of EvoNet being willing to take ownership of the problem despite the fact that these are customers whose business they paid for.

Of note is the excuse that EvoNet “has no lines” that they can manage “yet”. To manage internet services on an Openserve line, a company needs an ECS licence for regular connectivity or an IECS licence to include VOIP and physical line services in their product portfolio. These are awarded by ICASA, and you can’t sell internet connectivity services as an ISP without one (transit providers like Openserve, Internet Solutions, and Vumatel have an ECNS/IECNS licence, for nationwide networking). EvoNet does not have an IECS or IECNS licence despite the company offering VOIP service access to their clients on their website, and the company doesn’t appear in ICASA’s published list of ECS licence holders.

Crystal Web’s IECS licence was supposed to be transferred to EvoNet as part of the deal for the client book, according to Paul Hjul, which would allow them to manage internet services on leased Openserve lines. ICASA mediates licence transfers because they’re pretty pricey already. In a statement to NAG, Hjul said, “To the best of my knowledge no licence transfer has been effected.” Because there’s a dispute over the client book’s value and allegations from both parties that the terms of the agreement are being broken, that licence has not yet been transferred. I’m not sure it ever will.

It would be an interesting turn of events if EvoNet’s open-access network wasn’t running because they didn’t have an IECNS licence, because they expected to get an ECS/IECNS from Crystal Web. Let’s circle back to this point in a bit.

Scam alert?

On 30 October 2018, EvoNet sent out a notice to their clients via email that a scam had been doing the rounds claiming that the company would be changing their bank details:

Dear Client

It has come to our attention that some clients have received spam emails claiming that EvoNet will be changing bank accounts for payment purposes. Please take note that this is not accurate; does not originate from EvoNet; and as per our last formal communication, all clients are to please login to their client portal and only make payment from there (unless you are on debit order), including EFT payment clients where the option to make an instant EFT payment is available. We will not be disclosing a public bank account for manual EFT payments specifically to avoid this type of scam and we will not be accepting manual EFT payments in future made to any previous bank account that was in use.

If you have already made an EFT payment to an old bank account, please ensure that as of your very next payment that you instead login to your client portal and make payment directly from there using the various payment methods available.

Kind Regards
EvoNet Team

This is an interesting one because everyone that I’ve asked for proof of this email doesn’t have it in their inbox or their trash folder, and they don’t recall ever receiving it. Scams like this are pulled all the time, and with the leak of client email addresses from the 2016 breach still reflecting addresses used by real clients today it wouldn’t be surprising that scammers would be taking advantage of existing databases to find new targets.

If EvoNet’s messaging to their clients wasn’t already a little odd and unclear, they then confused their customers with an email sent much later on 20 November 2018, talking about banking details:

Dear valued clients,

URGENT REMINDER TO CHANGE BANKING DETAILS FOR INTERNET ACCOUNT PAYMENTS

As you know, EvoNet now owns the Crystal Web client base. As a result, you will need to change the banking details for payment of your internet connectivity each month, with immediate effect. Where you used to pay Crystal Web, you will now be paying EvoNet.

Our banking details are:

[REDACTED]

If you have automatic payments set up on your bank account for Crystal Web, these need to be changed to one of these two accounts before your next payment runs.

Please note that EvoNet cannot take responsibility for any payments made by mistake to the old Crystal Web banking details, and these cannot be transferred to EvoNet.

Thank you in advance for your kind co-operation.

The EvoNet team

Internet savvy users these days are taught to not trust emails ostensibly from companies they use that change their banking details. Sometimes those emails are legit, and are accompanied by other contact methods you can use to verify them, like a phone number. Although this email is legitimate, EvoNet gave customers no way easy to verify its authenticity, and no easy way to contact the company.

You could try call the number on their website, but it doesn’t work. You could try email them, but receiving a reply could take a while. You could try phoning Crystal Web, but to the average person this would be illogical because you’d be phoning Crystal Web and not EvoNet, and in multiple examples of support chats shared with me the EvoNet chat staff on the Crystal Web chat support keep on saying that Crystal Web is a separate company. To the average person, this makes no sense and almost looks like a scam.

But it gets stranger. This mail was sent out to customers on 26 November 2018:

Dear Customer

EvoNet have been working tirelessly with Sage (our Debit Order provider) to get the system up and running in order to get debit orders processed for the end of this month, unfortunately there have been some delays and we regret to advise that we are not going to be able to process debit orders this month end. Within the next month we will be sending out the New Debit Order Mandates that we will need signed and returned to us (PASA requirement) and thereafter debit orders will resume for the next billing period.

We appreciate your loyalty and apologise for any inconvenience.

Below are details of our new FNB and Nedbank accounts for your convenience

[REDACTED]

Regards

Evonet Team

Think back to last time, where customers were told in an email of all the myriad ways in which they could pay the company before the due date for their accounts on the first day of each month, but with the restriction that EFT customers needed to pay in two days before the end of the month, and one day before the end of the month if they could use credit card facilities or paid in using InstantEFT on the client portal. Any customer who had a debit order last month had to call up their bank and cancel it, and then schedule an EFT to go off two days before the due date at the end of the month, and then authorise another debit order for the start of the new year. It appears that the rift in Crystal Web that started in late September has reared its head again with EvoNet changing how they talk about money four times in two months.

EvoNet has also taken time to fix the branding on their invoices to customers, so that it reflects the correct company details. This new template was sent out to customers first in October, and the most recent version is shown below.

The invoices are still problematic, despite the correct branding and updated business details. EvoNet and Crystal Web now share the same telephone number, 010 612 0001, so calling EvoNet would take you to the Crystal Web call centre (and you still can’t call the company’s actual number listed on their site). It’s not a big deal because they’re the same company anyway, but branding is incredibly important for an ISP looking to stand out among the literally dozens of other competing companies. If your customers don’t know who they’re supposed to be calling, that doesn’t put them at ease.

The terms and conditions for customers at the bottom of the invoice on the second page is a dead link. The real agreement is available via this link. The agreement still says that the client “enters into an agreement with Crystal Web (Pty) Ltd”, which is incorrect because you’re doing business and receiving service from EvoNet, not Crystal Web. Legally, it’s a bit of a problem because Crystal Web has absolutely nothing to do with those customers anymore.

In addition, the address is now outdated as well because EvoNet has moved their offices. EvoNet included their address at 1 Avon Road, Ferndale for only two billing sessions before they moved offices to a new, undisclosed location. That address also needs to be updated for invoices that will be sent out for January 2019. Below is an excerpt of a chat I had with another support staffer at EvoNet on 27 November.

Wesley: Also, is the EvoNet office still at Avon Road? I tried calling the number on the website but it doesn’t work.
Support Major: Moving offices so space is still being fixed painting etc…..
Wesley: Stop huffing the paint. 😛
Support Major: lol We do what we have to man

I am also following up a lead on details regarding the lease of the company’s premises at 1 Avon Road. There are allegations of non-payment there as well.

This migration hasn’t exactly gone smoothly for anyone. Crystal Web lost all their customers and is essentially bankrupt, Crystal Web is in limbo because it doesn’t know what it’s supposed to be, and EvoNet doesn’t seem to have any inkling about what to do with the brand they’ve inherited. It’s a poisoned well. It’s a brand now mired in controversy and it begs the question of why they’re taking so long to dupe it.

Looking at the long term

All this raises the question of whether or not EvoNet has thought of the long game here. Two months ago, the company might have been in a position to purchase the client book from Crystal Web, pick up all their staff and branding, and still have finances spare to pay up the Openserve fees to ensure a smooth transition. We’re now in December, and as I’ve already reported there are many clients leaving the company as a result of the line suspension fiasco, not to mention customers who would leave anyway because they’d like to take a wait-and-see approach to a company that’s only two years old. That’s revenue lost for EvoNet.

Then there’s the revenue lost from not being able to build out their open access network because they do not have the proper licences, and revenue lost from resellers who would have been on their network but aren’t comfortable with how the company has communicated things so far (in addition to former Crystal Web resellers whose customers are leaving them as well). The company might have planned to have the relevant licence from ICASA under their control by now, but it hasn’t been transferred from Crystal Web by ICASA. Whatever the outcome of the Crystal Web saga, the fact remains that EvoNet was started with the idea of setting up and selling access to an open access, mixed-mode last mile network designed to offer fibre-like connections to consumers, backed by a rapidly expanding infrastructure. That kind of offering would have earned them millions of rands. Without that licence they’re unable to move forward, and with declining revenue they may have to scale back on the grand vision for a while.

The loss of the debit order mandates also makes things difficult. Companies model their finances with the expectation that a certain number of customers won’t pay on time, won’t pay enough, or won’t pay at all. Debit orders guarantee revenue straight from the banks, and a lot of companies in financial distress, or consumed by greed, see them as both lifelines and cash cows. Without guaranteed income, EvoNet is at risk. If they had to suffer another setback like a DDoS attack that lasts for more than a month, they’re not likely to recover from it.

By the time 2019 rolls around, EvoNet may find itself in financial difficulties. The irony in that statement is accompanied by a great sadness. The people who suffer disproportionately in this scenario are the employees in the lower rungs of the company, and its customers. The employees sitting at the desks manning the chat lines see with their own eyes how their job security is threatened as customers leave, cursing as they do because they’re frustrated. The customers at the other end are frustrated with the breakdown of trust – they gave the company money, and expected honest, good service which they aren’t receiving, and each time they look for an explanation the buck is passed off to another party. It may be months before everyone involved gets their dues and gets to move on.

The Saga continues.

More stuff like this: