Chain Reaction: How Supply Shortages Are Shaking Up The Channel

The burning question for solution providers today is how to navigate a broken supply chain that keeps putting obstacles in their way. Here’s how some have forged ahead and created stronger customer relationships in the process.

Mike Shook knew when the COVID-19 out­break wreaked supply chain havoc that it would change the fundamental makeup of his solution provider business.

He had already determined before the pandemic hit that his company, 5S Technologies, Cary, N.C., needed to transition to a cloud- and services-led model. But as hardware became more and more difficult to come by, it was the global supply chain crisis that lit the fuse and dramatically accelerated the solution provider’s transformation to become “software-heavy” as a matter of survival.

“When I place a software order, I have the license within 24 hours, and we’re paid within 30 days,” Shook, founder, presi­dent and CEO of 5S Technologies, told CRN. “With hardware, it could be six months to a year before you can deliver [with current supply chain backlogs].”

[RELATED: Supply Chain Issues ‘Driving Cloud Adoption’: CloudWerx CEO]

Shook is one of many savvy solution provider executives who pivoted his business to preserve revenue in the wake of supply chain challenges that have made it difficult to procure the hard­ware they typically used as solution building blocks.

“We took a long, hard look, and we changed who we were,” said Shook, who founded 5S Tech­nologies eight years ago as a small VAR focused on hardware sales. “We are now an MSP, and our managed services sell our software and hardware. We’ve gone full circle from the hardware being the horse that drives the cart to the software driving it.”

The scope of the two-year metamorphosis is evident in the numbers.

“Our VAR portfolio went from 80 percent hardware [and] 20 percent software to 60 percent software and 40 percent hardware,” Shook said.

During that time, 5S Technologies built up its hybrid cloud practice, boosting its focus on Citrix Systems and Nutanix, even becoming a Nutanix Champion Partner in August 2021. “It was easy for us to adopt that model once we got away from being a hardware shop,” Shook said. “Thankfully, we had those software technologies in place as people adopted work-from-home, and we were not relying on unknowns.”

At Meriplex, a Houston-based MSP, explosive growth in cloud and security services driven by the pandemic and the supply chain crisis has powered astronomical sales.

Cloud computing consulting and services revenue for Meriplex grew more than 50 percent in 2021 and is up another 50 percent this year. Security consulting is also on a sales tear, more than doubling each year of the pandemic.

The company’s hybrid solutions also have been up 50 percent year over year for the past two years.

“The pandemic accelerated the adoption of cloud computing,” said Meriplex Executive Director of Strategic Partnerships and M&A Neil Medwed, who has been at the forefront of a Meriplex acquisition binge that has brought 10 MSPs in the past two years under the Meriplex umbrella. “There are no ifs, ands or buts about it. Pre-pandemic, 25 percent of the customer base was deeply thinking about cloud. Because of the pandemic, 75 percent-plus of customers are thinking deeply about cloud and hybrid cloud solutions.”

Medwed estimates that about 40 percent of MSPs have made the transition to the cloud software and services model in the wake of the pandemic. That leaves a whopping 60 percent of the MSP base that has struggled and is not where it should be in terms of offering customers sophisticated cloud and security services, he said.

“If you stuck with a hardware-dominated model in the midst of the pandemic and sup­ply chain crisis, you are in the intensive care unit at this point,” Medwed said. “I believe that well-run MSPs are going to be able to take advantage of the market dynamics in a very positive way. Those com­panies that are not well run are going to struggle to keep up as days go by.”

Zac Paulson, CEO of TrueIT, a fast-growing Fargo, N.D.-based MSP, said the supply chain crisis has had an earth-shattering impact on nearly every part of his business. The resulting product shortages forced all of his customers to look deeply at cloud services, leading to a 20 percent to 30 percent increase in his cloud services business since the start of the pandemic.

“We were always a born-in-the-cloud-first type of company, but the customers weren’t always cloud-first,” Paulson said. “That conversation has gotten a lot easier. We were able to take the technology we use every day in our business and move it to our customers faster.”

TrueIT’s managed services business hit new highs in 2021 and 2022. “Our business is stronger than it has ever been because we were prepared to help our customers cross the chasm into the cloud,” Paulson said.

The supply chain crisis has changed the psyche of the channel regarding just-in-time manufacturing, said Paulson.

“We all got used to just-in-time inventory and project pro­cesses,” he said. “This has caused everyone to take a step back, do more planning, look at their forecasts and be a little more proactive. In all honesty, that is not a bad thing,” Paulson said.

Ultimately, the crisis has raised the status of MSPs acting as trusted advisers, Paulson said.

“Before the crisis, it was not uncommon for some of our managed services customers to buy online or go to big-box stores and buy stuff, and then we had to deal with the fallout,” he said. “When it became more difficult, they put the burden back on the MSP. The customers have welcomed that. I think they appreciate the fact that we are taking care of this now and helping them get through the supply chain crisis,” he said.

TrueIT has also begun holding inventory on certain hard-to-get products for customers. “We are holding laptops, desktops and access points,” he said. “Customers are purchasing product, and we are holding it for them until they need it. We never carried inventory in the past.”

Order Backlogs Still Persist

Marc Harrison, a 33-year veteran solution provider and PC reseller, wakes up every day with a knot in his stomach knowing he will spend countless hours on an increasingly futile attempt to chase down products for his customers.

“It makes you question why you’re in this business sometimes,” Harrison, president of Silicon East in Morganville, N.J., said with a sigh.

For Harrison, the transition to the cloud is not coming easily. He started offering more cloud-based services at the beginning of the pandemic “but for us that’s all been new business with existing clients. To date, no one has shut down their server infrastructure and never looked back,” he said. “The reality is, even with cloud-based apps and infrastructure, businesses still need PCs, switches, Wi-Fi access points, firewalls and printers. So the problem still exists for us.”

Harrison isn’t alone in his frustration. While the PC business saw an unprecedented boom in the two years following the onset of the COVID-19 pandemic, solution providers were about to get a crash course in the complex world of the global supply chain.

As COVID-19 pandemic lockdowns impacted just about every corner of the globe, a ripple effect would begin creating a sup­ply chain snarl that is still vexing industries around the world.

The IT industry took an especially nuanced hit, with vendors reporting record sales and demand entangled with constrained supplies and logistics delays.

While solution providers were busy trying to fulfill their backlogs, a perfect storm was waiting in 2022 as workers and students began returning to physical offices and schools. Demand began falling off deeply on the consumer PC side.

In addition, inflation spawned by global economic interven­tions was starting to rise. Russia invaded Ukraine in March, causing more logistical turmoil in the supply chain. And finally, COVID-19 reared its head again in China, this time striking some of the busiest technology manufacturing firms around Shenzen. Those market forces took a toll on financial results across the board in the tech sector and roiled the supply chain even further.

Networking, server and PC vendors have been hit particu­larly hard.

At networking giant Cisco, for example, the supply chain crisis has triggered a record-breaking order backlog to the tune of more than $15 billion as of May, up 130 percent year over year. The San Jose, Calif.-based company did not disclose those order backlog numbers for its most recent fiscal quarter that ended July 30. But the company once again saw triple-digit backlog growth that fueled the backlog to the highest level ever recorded in the company’s history, Cisco Chair and CEO Chuck Robbins said in August.

Hewlett Packard Enterprise’s backlog, meanwhile, hit yet anoth­er record level during its third fiscal quarter ended July 31, up 96 percent year over year. The backlog includes Aruba order bookings of more than 20 times historical levels and a backlog in the compute business that also hit another record level and is now at five times normal levels.

The backlog also impacted Spring, Texas-based HPE’s grow­ing GreenLake cloud services business, which was “limited” by supply chain constraints that hindered the ability to do some GreenLake installations in the quarter, according to HPE Presi­dent and CEO Antonio Neri.

The supply chain delays are impacting solution providers in different ways. Smaller players seem to be bearing the brunt of backlogs for ready stock, solution providers said. Larger compa­nies with more purchasing power can hold more inventory. For Silicon East’s Harrison, the difficulties in the ordering process create a constant supply headache.

Harrison explained a typical problem with ordering a server for a customer: “We buy these large, $20,000 to $25,000 servers,” he said. “The server might be in stock, but there are no hard drives and no SSDs [solid state drives] to go with it. In some cases, there are no power supplies. And you need all those components to deliver a solution. The problem is that the power supply might be back-ordered, say six weeks … the hard drives are back-ordered 12 to 16 weeks, and for the SSDs, there’s no delivery date at all. So, you take delivery of the server and pay for the other parts as they become avail­able. And there’s no guarantee the hard drives, for example, will ever come in.”

Some customers, he said, have been waiting more than two years for server parts to complete projects. “This is making it impossible to provide the solutions customers have paid for. … It’s ridiculous,” Harrison said.

As remote work and remote learning fueled a rush to equip workers and students with PCs and networking devices during the pandemic, Harrison was suddenly flooded with demand while he watched supply for these products get increasingly more difficult to secure.

Network switches, for example, have become a nightmare to deliver, he said.

“On the network side, we’re going through our hundreds of installations and trying to figure out where to put in switches that may be larger ca­pacity than we actually need—but that’s what’s available. We’re literally playing ‘musical switches.’ It’s just not sustainable in the long term.”

Wait times for some hardware products are lasting nearly a year right now, accord­ing to Hermann Masser, president of Masser Technologies, an El Paso, Texas-based solution provider. The issue is particularly bad for networking gear, he said, which has prompted Masser Technologies to stock used equipment to fill in temporarily as needed.

“We’re keeping an inventory of used equipment so when one of my customers that is waiting to get the hardware, or when they have a failure and they have to replace a switch in an emergency because we cannot find switches or access points anymore, we’ll grab one of those old pieces of equipment and deploy it as a Band-Aid until we can get the equipment back,” Masser said.

Masser’s company is waiting on close to 200 access points right now—products he doesn’t expect to have in hand for another year. “It’s definitely changing our ability to move fast and to sell,” he added.

A solution provider who spoke to CRN on condition of ano­nymity said that the network switch shortage has contributed to a 30 percent decrease in hardware sales for his company this year compared with last year.

The Texas-based solution provider has about 60 SMB custom­ers and has been forced to use older network switches to patch their networks as the wait for parts continues.

In addition to networking products, storage products have been hard to find, said TrueIT’s Paulson. In fact, he said, some customers that ordered products in January are just now receiv­ing the equipment, he said.

“Every customer that needed networking or SANs faced delays,” Paulson said. “There were none that we could order and deliver in a normal time frame.”

TrueIT also had orders canceled by distributors that were fac­ing one delay after another from vendor partners, said Paulson. “Distributors thought they would have stock coming in, but the estimated time of arrivals just got way too long so the orders were canceled,” he said.

TrueIT also saw big price increases, which caused the solution provider to go back to its customers with 10 percent to 30 percent price increases, according to Paulson.

Ultimately, True IT de­livered fewer hardware products to customers but at a higher price, with the result that overall hardware sales have risen, said Paulson. “Hardware quantity is down by a sig­nificant number, but our hardware sales overall are up,” he said.

The ‘Humpty Dumpty’ Supply Chain

For Harry Zarek’s Ontario, Canada-based solution provider Com­pugen, the race to outfit a tech-heavy workforce to operate from home during the pandemic was a wake-up call to the channel that it had long overlooked the complexity of the supply chain.

“It was an awakening about how dumb and stupid we all were about the supply chain because it works so well, right until it falls apart,” said Zarek, founder, president and CEO of Compugen. “And in fact, it’s so broken, it’s like Humpty Dumpty. But there’s no way any human being could have seen it coming. People were sending me pictures of themselves trying to cobble together home computers, old laptops, and trying to assemble some sort of way to work from their kitchen or basement. The first part of it was just pandemonium.”

As the pandemic wore on, the challenges shifted, he said.

“We went through phases,” Zarek said. “The first phase was this mad scramble to get anything that customers could use to allow them to maintain operations. Then there was this short period of time where businesses seemed to go into suspended animation—they just paused. Then, business began to acceler­ate. Every month it was more and more, and as business kept growing the supply chain issues kept growing along with them. It’s a never-ending game of Whac-A-Mole.”

The Power Of Global Logistics

When Future Tech Enterprise CEO Bob Venero made a bold bet on building out global logistics capabilities for his solution provider business a decade ago, he had no idea it would reap big rewards in the wake of a global supply chain crisis.

“Global logistics are on the forefront of every meeting and conversation we are having with customers and OEMs,” he said. “We are tracking this weekly with our OEMs. Every week we have supply chain calls at every level for every product for every customer that we sell to.”

With its global logistics capabilities, Future Tech’s business is “booming,” with sales up 49 percent in the last year, said Venero. “Our hardware business, including mobile, desktop, server and storage systems, is extremely healthy,” he said.

One reason for the uptick in business is the ability for Fort Lau­derdale, Fla.-based Future Tech to leverage its finan­cial muscle to gain a sup­ply chain advantage for its large enterprise customers by increasing its inventory levels from three months of inventory before the pandemic to six to nine months of inventory, said Venero.

“Our job is to make sure we can meet these supply chain challenges for our customers,” he said. “We have changed and adjusted the way we do business with increased inventory levels for the last two years. We are holding up to $100 million in inventory today, up from only $30 million pre-pandemic.”

But Venero stressed that a global logistics buildout is not for the faint of heart.

“Eighty percent of the pain we have as a company comes from the global work that we do for our customers,” he said. “Global logistics is extremely tough. You are talking about deal­ing with every country, every region, every OEM, every rule and regulation in these countries, every certification. Take what you have in the U.S. and multiply that by the 47 countries that we operate in today and all the OEMs that we work with and you can understand the level of complexity that we are dealing with as a global logistics company.”

Fending Off Bad Behavior

Rhino Networks, an Asheville, N.C.-based solution provider and Cisco partner that specializes in Cisco’s Meraki portfolio of cloud-based products, serves thousands of customers in the U.S. It teams with distributor Ingram Micro for Cisco Meraki product fulfillment, a process that has worked well for Rhino Networks for the past decade. That’s because Cisco Meraki products have always had very short—nearly nonexistent—lead times, offering an “Amazon-esque” experience for customers, according to Todd Carriker, founder and CEO of Rhino Networks.

But the days when orders would arrive within 48 to 72 hours are long gone, he said.

Early on in the supply chain crisis, the long and disruptive lead times for gear sparked “massive amounts” of disbelief from customers. “[They would say], ‘That can’t be true. Surely that’s not the case.’ Or they’d say, ‘Well, instead of doing Cisco-based wireless access points, I’m going to go buy Aruba or Ubiquiti,’” he said.

Then, the bad behavior started. Customers began calling multiple Cisco partners in an attempt to get different answers. But businesses quickly learned that all solution providers were in the same boat. Customers also began placing product orders across several different vendors. When they would receive a tracking num­ber for shipping, they’d cancel the other orders, which further threw off inventory allocation and forecasting for nearly ev­ery major manufacturer, Carriker said.

Out of desperation, some customers began dipping into the gray market and buying gear without a warranty and with potentially compromised serial numbers. One business bought $150,000 worth of gray market Cisco gear that was largely unusable, Carriker said. “We’ve seen people be burned,” he added.

Rhino Networks immediately began heading off these issues at the pass by being as transparent as it could with its custom­ers about shortages and lead times to set realistic expectations. The company pivoted on a dime and started showing customers other options outside the Meraki portfolio that had shorter lead times. For example, if a customer was interested in a Meraki access point, another comparable Cisco access point might fit the bill based on the customer’s deployment and consumption plans. And if a customer knew it would be ordering more a few months down the line, Rhino Networks would put together a bulk order to get the company a discount, Carriker said.

“Essentially, if they could put some skin in the game and invest a little more, we’d help out. It’s about being creative and showing people you care,” he said.

For Calgary, Alberta-based Cisco partner Long View Systems, the biggest challenge at the start of the crisis was fluctuating shipping times that would often change in “the 11th hour,” creat­ing uncertainty for solution providers and end customers, said Lane Irvine, the company’s network business solutions director.

“Early on, we would see ship dates for products that were supposed to show up three days from now, but then the ship date would change to three months out,” Irvine said.

Products like the Cisco Catalyst 9200 switch series were not available, while others had better availability, such as Cisco’s Small Business 300 Series switch, Irvine said. “We had to do a review and update of what products we should be ordering, and we had to work with clients and make the shift in a number of cases,” he added.

The solution provider also worked with frustrated customers by staggering product deployments. For projects that may have required 300 new access points, for example, it was deploying 20 new access points every week. This “trickle” allowed customers to make progress on their IT goals, said Kent MacDonald, senior vice president of strategic alliances at Long View.

“It was much more manageable to start projects in bite-sized chunks rather than having to wait for everything to arrive to start the proj­ect,” MacDonald said.

At the same time, Long View’s cloud and man­aged services businesses have flourished.

Long View has been building a practice around the increasingly popular Everything-as-a-Service (XaaS) buying model for its customers. The solution provider this year became the first Cisco Gold Integrator and Gold Provider partner in Canada with a Hybrid Cloud Cisco Plus deal, Cisco’s XaaS portfolio. Long View’s own as-a-service cloud infrastructure, On-Demand Infrastructure, is built on Cisco technology.

“With XaaS, you don’t have a supply chain or shipping delay issue because the capacity is instant-on,” MacDonald said. “The supply chain issue really does help punctuate the value of XaaS, especially in a data center infrastructure environment.”

Forecasting Becomes Crucial

Mike Turicchi, vice president of Gainesville, Va.-based NCS Tech­nologies, said the big changes at the beginning of the pandemic were a shock to the system, but his company upped its game when it came to forecasting and communication with custom­ers. “It came on pretty quickly,” he said. “We have about 400 [vendor] partners in our ecosystem, and in the beginning, the top 50 or so were reaching out quickly saying, ‘You might want to put some orders in. Prices are going to go up. Supply is going to get tight,’” he said.

“To some extent it was out of our control—these changes were coming, whether we liked it or not. We’re not like Apple where we could go out and buy a bunch of inventory and ware­house it somewhere.”

Turicchi said forecasting and communication with customers have become crucial since the pandemic first hit.

“It’s very much a cat-and-mouse game where you’re asking the customers what’s going on, and you’re telling them what’s going on from your side,” he said. “It’s become more about building relationships.”

Solution providers have had to learn to forge stronger bonds with customers and with vendors, Turicchi said.

“It was a chance to really get to know the customer more intimately and get some more insight,” he said. In the past, solu­tion providers may have gotten used to a more turnkey process of buying and selling, Turicchi said. “We were all used to that … push-button aspect of supply. So, we were all pretty demanding of the vendors. We had to step back and say, ‘We need these guys more than ever, so we need to play a little nicer.’ Price isn’t everything to­day. That’s been a huge shift,” he said.

But even as the busi­ness shifts and refocuses, Turicchi said the relation­ships built during the pan­demic will have a lasting impact on the channel community.

“At the end of the day, we all want to make money, but the relationship aspect has become a big ingredient,” he said.

The Cloud Needs To Be A ‘Driving Force’

Meriplex’s Medwed believes the pandemic and supply chain crisis have forever shifted the market dynamics from a channel model focused on hardware sales to a cloud-dominated model.

“Before the pandemic, cloud was an important component of the average MSP portfolio, but it wasn’t the featured component of the portfolio,” he said. “In this day and age, cloud needs to be a highly featured component of the portfolio. It has to be a driv­ing force of your expansion because clients are moving toward hybrid models. The days of how much hardware I can sell and at what margin are behind us.”

5S Technologies’ Shook said his company would be in serious trouble if it were still focused primarily on hardware.

“People on the hardware side are trying to address the supply chain problems with Band-Aids and hope it gets better. They are doing what they can to stay alive. If all I did was sell hardware, we wouldn’t be able to pay the bills right now. I’d imagine anyone in that position is struggling.”

STEVEN BURKE contributed to this story