The Clock Is Ticking: Cisco Exec On Component Failure And Why It Won't Reimburse Partners

Major Problem For Cisco

Jennifer Ho, manager of Cisco's Business Critical Communications, spoke with CRN about a massive product replacement initiative impacting mission critical Cisco products including ASA firewalls, ISR routers, Nexus switches and Meraki cloud-based managed switches.

The "Clock Signal Component Issue" causes a number of Cisco products to fail after 18 months with the end result being "the system will stop functioning, will not boot and is not recoverable."

Cisco said the clock signal component inside certain devices degrades over time. "Although the Cisco products with these components are currently performing normally, we expect product failures to increase over the years beginning after the unit has been in operation for approximately 18 months," said Cisco in the notice posted on its website.

Does Cisco have any reimbursement plan for channel partners who are currently sending out engineers and systems integrators to uninstall and replace these faulty devices?

Because the product doesn't actually have any issues for 18 months, we really wanted to focus our funding for the issue on a replacement program. So we're proactively offering the replacement of products even before the product fails, and we really believe that this focus on providing the best-quality products is the right focus for our Cisco customers and the partners' customers.

Unfortunately, because our funding is focused on providing the products, we are unable to reimburse for on-site services to replace the affected devices. Customers may have field engineering service as an option for their services contract, in which case the field engineering support would be included with the replacement.

So if channel partners are providing their own services on top of Cisco products, there's no reimbursement funding for them right now if they go out and replace these faulty devices?

That's correct. We're focused mostly on the product replacement side.

For our customers, if it's a fix-on-fail and that is included in their service contract, then we would certainly support that versus the proactive product replacement. … 'Fix-on-fail' means that if the product has already failed – i.e., it has stopped operating and will not reboot – and the customer has service replacement in their contract, we will honor the terms of the contract.

Can you give us more insight about this replacement program funding?

The funding for this replacement program is being used to proactively offer replacement products, even before the product fails, to help our customers reduce risk from their environments. We believe this focus on providing the best-quality products for all Cisco customers is the right one. However, due to this, we are unable to fund or reimburse for on-site services to replace potentially affected devices. This is consistent with our standard policy of addressing hardware failures and replacements.

Partners said there are some delays with obtaining specific products due to the high demand. What is your advice for Cisco partners to combat this?

We're working quickly to respond to needs and we are prioritizing a replacement based on the product's time in operation. Because of this prioritization, fulfillment request could be delayed. So I'm sure [partners] may start seeing that. They could be fulfilled in multiple shipments .… Understanding that there may be some delay, particularly if it's not a product that's been in the field at that 18-month time frame or longer, those may be delayed or fulfilled in multiple shipments.

What does Cisco mean by multiple shipments?

If [a partner] puts in an order for a bunch of shipments, we may be able to replace some now and then some a little bit later. We'll try to replace what we can as quickly as we can, then fulfill the rest of the shipments potentially at a later date. So the shipments may be split up.

Is there any priority to replace one faulty product faster than another -- such as, is Cisco trying to replace ASA security firewalls faster than Nexus switches?

There's not a prioritization based on the product. Specifically, we are prioritizing based on the time in operation. So really looking at trying to replace those products that are hitting that 18-month period or longer in operation.

Cisco knew about the problem since November 2016. What has Cisco done over the past few months?

It was late November when we became aware of the issue. In that time, we had to work very closely with the supplier to identify the exact issue, work on creating a fix for the issue and then testing that fix with our products. So there's a lot of work on that end. We also worked with our executives to create our remediation plan for our customers.

As far as getting the word out to customers, when did that process begin?

All of our customers were alerted to the field notice process with the information that was posted on Feb. 2, [2017], and our customer account teams have been reaching out to those customers believed to be affected. So that process is all in the works now.

So these affected products have been taken out of rotation and the products currently being shipped now are issue-free?

That's correct. Anything shipping now does not have the faulty component.

How long have these products been shipping issue-free?

The shipping date of the products with that fixed component is actually different by each product. On our ordering form customers are able to fill in the serial numbers of their products and that form will determine if that product is affected or not. … It is different by each product, so it's best to check on the ordering form.

What other type of channel aid does Cisco plan on rolling out over the next few days and weeks to help partners through this issue?

We've certainly prepared our internal team to ensure that they fully support our channel partners and make sure they're in the loop. We do have information posted for our partners with an overview, FAQs and there's some analytics to help them feed more visibility into their install base. We have some analytics tools that partners can use to make it a little bit easier for them.

What type of analytics tools?

It's a tool partners can use to discover what their customer install base is for these products. It's a link. … It's not really a product that we have, but it's a tool that they can use. We've provided analytics to give more install-base visibility to our partners.