This whitepaper identifies the common elements of modern architecture that are likely to remain the same going forward and to define how they relate to a cable management plan.
It highlights considerations that will support those changes and allow data centers to respond with agility.
Contents
Enter your details below to download the Whitepaper PDF:
Modern data center architecture is changing at incredible speeds. Even a basic understanding and overview of data center architecture evolves rapidly.
However, a general, foundational understanding can be achieved through looking at structure overall, and the common elements that remain the same even as the applications change.
Cable management is a large part of that picture. Developing agile cable management plans is just as important, if not more so than the rest of your data center architecture. This whitepaper identifies the common elements of modern architecture that are likely to remain the same going forward and to define how they relate to a cable management plan and considerations that will support those changes.
1. The Importance of Cable Management
Effective cable management allows technicians to simply unplug and plug in cables that are necessary to the action they are taking. Disorganized cables may involve “troubleshooting” steps that really involve determining what is plugged into what before work can begin.
Effective airflow prevents overheating, fire hazards and even cable breakage that can cause serious issues – or at worst, an outage.
Cables that are too long and disorganized can also quite simply be a tripping hazard, increasing the risk of workplace injuries. In addition, there is a risk of electric shock.
With the internet of things, the rise of VR and AR applications, and even the adoption of 5G networks, speed has become ever more critical, and will continue to be so over time. That means you need to have the ability to add to your server stacks quickly and easily.
But congested cabling can make this problematic. You’ll have to untangle and organize what you have before you can safely add more. A simplified, organized and effective cable management system is an important element of IT infrastructure scalability, and will make future expansion that much easier.
That means putting some thought into planning.
1.1 Intelligent Infrastructure
The most important aspect of any infrastructure is that it needs to be intelligent. Artificially and humanly intelligent. Software, machine learning, and artificial intelligence all contribute to a hybrid infrastructure that can adapt and scale according to your data needs.
This intelligence exists outside the type of structure. Whether we are talking about cloud data management, physical data centers, a hybrid approach, hyperscaling, collocation and all of the data types out there, all share this common trait. This intelligent infrastructure comes from not only understanding the data center trends of both today and tomorrow but from the application of that understanding.
With that in mind, let’s take a look at current trends and architecture, and then we can tackle how cable management supports this modern data center architecture.
1.2 Hyperscaling and Colocation
There are two clear trends in data centers, and they have a clear relationship. The first is hyperscaling, the act of a company or group creating data center space on a large scale and in a number of locations. Essentially, this data center space then becomes available for “rent” or rather collocation.
Colocation is about more than just “renting” data center floor space. It is also about putting your servers where you can take advantage of available computing power, and where you can cooperate with other “collocation partners” to increase your overall computing potential.
Collocation is really a move to “share” space rather than building your own data center either because your needs are too small to justify your own space, or because you know you will need to scale at some point, and collocation is one of the easiest ways you can prepare for that future.
For small to medium businesses, this method in combination with cloud database storage seems to work really well, but it has also seen success at the enterprise level. The idea of combined computing power and scalability has a large appeal for many types of businesses.
But there is a challenge here. Many “collocation” companies are still more real estate or space rental agents rather than “capacity and power” agents. While that mentality is still shifting, it’s one of the primary concerns data center managers have moving forward.
“Workload placement is not only about moving to the cloud, it is about creating a baseline for infrastructure strategy based on workloads rather than physical data centers,” David Cappuccio, Distinguished VP Analyst; Henrique Cecchi, Senior Director, Gartner Research Report said. “This is causing I&O leaders to rethink infrastructure strategies, which have a direct impact on enterprise data centers.”
1.3 Constructing Your Cable Management Plan
The basic tenets of cable management are well established, most likely known to you already, and are tried and true over many years. Planning Effective Power and Data Cable Management in IT Racks by Schneider Electric provides a good overview on power and data cable management, that it says “will improve physical appearance, cable traceability, airflow, cooling efficiency and troubleshooting time while reducing the risk of human error.” They recommend the following steps:
- Plan
- Determine the routes for power and data cables
- Identify cables
- Route and retain cables
- Secure cables and connectors
- Avoid thermal issues
- Document and maintain organization
We concur with the approach they outline, and rather than cover the same in this whitepaper, we ask instead – what’s changed in the cable management space? What new technological advances have also transformed cables and cable management? And how can cable and cable management considerations support a more agile data center, in preparation for future advances in the field?
So what are network architectures as they are now?
2. Network Architecture Types
There are several different types of network architecture, and each has pros and cons, and its own place in the current development of data centers. Here are a few of the most common, and how they work.
2.1 Three-Tier
The traditional data center architecture is a three-tier one. While this system offers a lot of redundancy, and therefore data protection, it has in some ways become outdated. The reason is there are latency and power drawbacks, so as the need for speed increases, the popularity of this structure declines.
Essentially, servers connect to access switches, which then connect to aggregate switches and finally to core switches (the three tiers). There are active connections and inactive backup cabling for redundancy.
This type of structure often uses fiber cables, both MTP 12 and -24 between core and aggregate switches. LC cabling then runs to the access switches.
Three-tier network structure does have some advantages. Various patching systems allow for rapid deployment, moves, and changes, and the right cable management system can accommodate a variety of density options.
However, some newer structure types offer additional benefits.
2.2 Spine Leaf
When compared to the traditional three-tier system, the spine leaf network structure has some latency and power reduction advantages that are worth exploring. This “fat tree” infrastructure features several paths between connection switches and access switching. This helps cluster powerful computing power only using two-tiers rather than three.
This structure also uses fewer switches overall and reduces the number of redundant paths, which facilitates the power and latency advantages. This has become a common structure for greenfield data centers. Cabling is using specific fiber connections and proper management can also enhance density. This is a very scalable structure, and that scalability is another reason it has become so popular.
2.3 Switch Fabric Architecture
When we talk about switch fabric architecture, we are actually talking about a few different types of both cabling and setup. Each has it’s own application, but all are included in the ANSI/TIA-942-B published standard. Like the Spine Leaf Architecture, these are still emerging and evolving.
Switch fabric network architecture includes: full mesh, interconnected mesh, centralized and virtual switch. Let’s look quickly at each one.
- Full Mesh: Each switch is connected to every other switch. The disadvantage is this is not very scalable. An equipment distribution area (EDA) is noticeably absent as a fabric for Top of Rack (ToR) topology. Best for small companies or data centers.
- Interconnected Mesh: Similar to full mesh, but there are a few interconnection switches, which makes it more scalable and is good for growing companies or data centers.
- Centralized Switch: in this case, the servers are connected directly to switches in almost a single tier. This is easy to maintain, even with a small staff, but does not scale well because there are only so many ports to connect to.
This architecture is, at the moment, best for small data centers who may not need to scale rapidly, with the exception of a few interesting applications of the interconnected mesh systems. The full mesh and centralized switch architectures, besides not being as scalable as others, also present density challenges, some of which can be addressed with the right cable management and server plans.
2.4 Centralized Patching
Centralized patching is used with two tier systems such as the spine leaf method. Essentially, the idea creates a central switching area, bringing more fiber into what is known as the main distribution area (MDA). This area can handle all cables from the data center in a central area. Maintenance for larger data centers is simplified.
In addition, moves, adds and changes (MACs) are much simpler as a result. These central areas do not have to be in the center of the data center to function well, but can be a separate area allowing you maximize your prime data center floor space.
As far as cable management goes, some data centers are adopting frames in this center area rather than traditional server cabinets, and this approach requires a unique cable and power management strategy. Once established, this strategy does help with maintenance and scaling in the data center.
This is by far not a comprehensive list of data center network infrastructure types. New ones are emerging as the size and shape of servers themselves changes, new cable types, sizes, and connectors develop and come into common use in data centers.
What are these cable types and what do they mean to data center management?
3. Cable Management Plan Considerations
Now that we have an understanding of the overall architecture, we’ll cover the considerations you should keep in mind as you develop your cable management plan. Rather than re-create here, we recommend the blog posts on the following topics. (Subscribe to our Newsletter to stay on top of these and other data center shifts).
3.1 The Changing Cable Types
There are a number of cable types used in any data center, and they vary from communication cables such as fiber cables to power cables. All of these cables must be managed properly while still allowing for cooling airflow and a labeling system. This labeling system is also a larger part of the overall data center management plan.
First, let’s look at the types of communication cables:
- Fiber Optic – High-speed, high bandwidth, short and long-distance communication cables. (LC, SC, FDDS, MTP, MTRU, FC, etc.)
- Fiber Channel – High-speed twinaxial or fiber cables (QSFP, SFP+, 10G, CX4, LC, SC, ST
- 40 or 100 Gbps Ethernet – Cat7A Cables, GG45, TERA
There are also other Gigabit Ethernet cables of various types. Current trends indicate a move toward shorter distance cables that carry more information, larger servers themselves, and a reduction in the number of connections where latency can result even when longer cables are used.
Connectors are also evolving in ways that offer security and protection from accidental disconnection and other common issues.
Some data centers still use copper cables for some applications, including power cables. They are less expensive, although they require more shielding than fiber cables. Copper also requires less cooling, so when used in conjunction with other cables, significant power cost savings can result. However reliable and consistent they are, copper cables don’t measure up to fiber in some key applications.
Fiber cable is lighter and thinner than copper and is improving all the time. Because of the optic and connector advancements, data can be carried over a longer distance, and therefore works for connections inside the rack as well as cabling racks together.
Bend radius is another concern, and generally speaking, thinner cables can bend more easily, although the connections with thicker cables can be more secure and less fragile. It all depends on the application of each.
Power cables are pretty straightforward, but power cable management can be a little more flexible. From underfloor cabling to overhead cabling, the pros and cons depend upon the overall data center plan, scalability, and the specific servers, racks, and cable management used. Ground cabling presents its own challenges and needs its own plan.
Keep in mind as data centers become “greener” power transmission from solar, wind, and other sources present their own cabling challenges. The one thing we can know for sure about cable development is that it will continue to change and evolve as data centers meet new demands and challenges.
Recommended Reading:
4 Cable Improvements That Increase Data Center Efficiency and Build Scalability
The data center of the future is needed now. Added to rising growth in a world lived increasingly online, augmented reality (AR), virtual reality (VR) and 5G are driving greater demand for data speed and volume. How can you ensure your data center is operating at peak performance now – and has the capacity to sustain performance as demand grows? Building for scalability is the key. Fortunately, those elements that create data center efficiency now lay the groundwork for your ability to respond well in the future.
Proper Power Cable Usage Prevents Poor Performance
Data centers are moving away from under-floor cooling and cabling, and in many cases to overhead cables and more advanced cooling systems. At the same time, what may seem the simplest of things, power cables, are evolving and changing as well. The reasons are quite simple. New angles, new cable types, and even new connection types are designed to ensure maximum uptime, no cable failures, and the prevention of disastrous accidental disconnects or power loss.
3.2 Cable Management Options
Modern data centers use a structured cabling plan with labels, an overall routing plan, and standardised plans for moves, changes, and additions. Cable management will largely depend on the type and number of cables you are using, overhead or underfloor routing, and specific airflow and cooling requirements.
The denser your floor plan, the more attention the cooling plan will require. At the same time, using ZeroU cable management systems, the correct racks for your application, and the best cable options for the distance and data transmission you need enables you to use the space you have to the greatest efficiency.
When it is time to make additions and scale, planning for this extra space can be critical. 22RU vertical cable management allows you to free up both rack and floor space, even when you are in the early planning stages.
Architecture is all about balancing density and the use of space with cooling, the capacity to scale, and the ability to utilize computing power as efficiently as possible.
Recommended Reading:
8 Critical Data Center Practices for Floor Design and Delivery
The physical layer of the internet, the data center, is largely dependent on floor plans, not only the floor design and type of the floor itself, but where you put everything, and how that impacts data accessibility and delivery for your clients. Perhaps the most important feature of any data center is agility and flexibility. To prepare for the future, floor plans must have the ability to adapt built-in. How do we get there?
Dead Servers Walking – Zombie Servers and Data Center Efficiency
Data centers use a lot of power, and while the move to renewable energy helps, other efficiencies must be created to keep these power-hungry behemoths in check. It is estimated that 2% of the carbon emissions in the world come from data centers, and that will only increase as we stream more, save more on the cloud, and demand internet that is faster and faster. However, there is a monster or monsters lurking in nearly every data center – zombie servers.
How Cable Management and Airflow Management Impact Each Other
Here’s the simple truth of the matter. Cable management can impact airflow management , and airflow should definitely inform the method, type, and execution of cable management. To understand how the two impact each other and what the best overall solution will be for your data center, it is important for use to take a look at cable management and airflow from a high level perspective.
3.3 Cable Management Racks
Server Rack Configuration and Cable Management Best Practices
There are only three types of currency in the world: time, money, and expertise – and we can’t afford to waste any of them. You have the expertise needed to run a data center and you hire others with the same expertise. But there are two things you can always save in your data center: time and money.
Rapid Data Center Growth: Keys to Rack Management Success
The COVID pandemic hit, and workers headed home for good, but not just to binge old episodes of Fringe and eat ice cream on the sofa. They headed there to work, which meant that suddenly data centers were overloaded. Operating at near maximum capacity, HVAC systems strained to keep up, and data center managers lost fistfuls of hair seeking solutions to keeping up with the exponential pace of data center growth.
3.4 Cable Labelling
How to Design an Effective Data Center Cable Labeling System
One of the things we talk about often in cable management besides having the right cable management and rack management systems that make your data center the most efficient, is using an effective cable labeling system.
3.5 Monitoring and Remote Control
Artificial Intelligence in Data Centers: The New Reality
For years, artificial intelligence, or AI has been the topic of science fiction, from Skynet in the Terminator movies to more benevolent systems. But AI has now become a reality, and that reality has put artificial intelligence in data centers, and the trend is growing. Data centers employing this technology are often referred to as enhanced data centers.
Using Data Center IoT to Automate and Improve Operational Efficiency
In order to meet demand, data centers are facing the need for physical expansion of their capabilities, such as the addition of new server racks, greater capacity HVAC systems and more. The problem is that those needing to making physical changes cannot do so remotely. Fortunately, there is a solution, which combines traditional data center infrastructure management (DCIM) with the application of the Internet of Things (IoT). Once implemented, data center IoT can help minimize the need for on site interventions, reduce costs and enable better data collection.
4. The Future of Data Center Architecture
So what does the future of data center architecture look like? Well, to simplify it, we can’t predict all the coming changes. What we can say is there are underlying principles that, if in place, will help any data center be prepared for tomorrow.
4.1 Good Bones
Underlying structure is essential. The right rack placement, strategic placement of a centralized patching area and MDA all set a data center up for ease in moves and changes, additions, scaling, and more. Cables can be replaced or upgraded, as can equipment, and as long as the underlying structure exists, this can be done simply and easily.
4.2 Adaptive Designs
Good bones leave to adaptive designs: designs that can change and flow with advancements in technology without being left behind, and enable technicians and managers alike to adapt as needed. There is no doubt that new server technology and cable advancements will come.
Will the next group of servers be the 23” cases we are seeing now? Will they move back to a 19” size? Will cables get thinner and shorter, or move to longer cables with more shielding?
Only time will tell. Making sure your design and data center plan is adaptive will ensure that you are prepared either way.
4.3 Overall Layout
Lastly the overall layout of the data center must be ready for tomorrow. Everything, including cooling and power must be scalable. The flooring plan and the flooring itself must be adaptive in nature. These things must be determined with the future in mind, and with the idea of supporting an evolving cable management plan.
4.4 Green Machines
As society’s demand for data and the forces of climate change continue to grow, so too will the demand for green data centers and their focus on a sustainable future.
Companies intent on lowering their carbon footprint usually have two major energy goals for their green data centers: (1) reduce the total energy usage of the centers, and (2) increase the amount of renewable energy used by those facilities.
A metric commonly used to measure the energy efficiency of a data center is called its power usage effectiveness (PUE), a ratio of its total power consumption divided by the amount of power consumed by its computing infrastructure. PUE was developed by the Green Grid, a consortium of policymakers, technology providers, facility architects and utility companies dedicated to improving IT and data center efficiency.
In a perfect world, PUE would be 1.0, meaning that all of the power consumed by a data center is being used to power its servers. In reality, however, a data center must also provide power to cool its computers, and maintain other aspects of its infrastructure such as facility lighting, heating and cooling. According to the Uptime Institute, the average PUE for data centers in 2019 was 1.67.
The goal is that through modern data center architecture, PUE is decreased, and over time we will get closer to that “perfect world.”
Keep in mind as data centers become “greener” power transmission from solar, wind, and other sources present their own cabling challenges. The one thing we can know for sure about cable development is that it will continue to change and evolve as data centers meet new demands and challenges.
5. Wrapping it Up
Cable management supports modern data center architecture in a variety of ways, but only when that architecture is created with cable management integrated into the plan. The right cable management plan can increase the capacity for density, preserve valuable floor space, and make moves and changes with ease.
6. Glossary
| ANSI/TIA-942-B STANDARD A telecommunications infrastructure standard for data centers. An American National Standard (ANS) that specifies the minimum requirements for structured cabling work defined in Telecommunications Industry Association (TIA) standards |
| ARTIFICIAL INTELLIGENCE Intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals |
| BEND RADIUS Measured on the inside curvature, the minimum radius one can bend a cable without kinking it, damaging it, or shortening its life |
| CABLE MANAGEMENT PLAN Management of electrical or optical cable in a cabinet or an installation |
| CABLE MANAGEMENT SYSTEM Element of IT infrastructure that relates to the management of cables |
| CENTRALIZED PATCHING Creates a central switching area in two tier systems |
| CENTRALIZED SWITCH Servers are connected directly to switches in a single tier |
| COLOCATION Shared data center space |
| CONNECTORS Electromechanical device used to join electrical conductors and create an electrical circuit |
| DATA CENTER ARCHITECTURE How all data center resources are interconnected |
| DATA CENTER FLOORPLAN Layout of the boundaries of the room (or rooms) and the IT equipment within |
| DATA CENTER PHYSICAL LAYER Transport of data using electrical, mechanical or procedural interfaces |
| ETHERNET CABLE Network cable used for high-speed wired network connections between two devices |
| FIBER CHANNEL CABLE High speed twinaxial or fiber cables |
| FIBER OPTIC CABLE High-speed, high bandwidth, short and long distance communication cables |
| FULL MESH Each switch is connected to every other switch |
| HYPERSCALE Company or group creating data center space on a large scale and in numerous locations |
| INTERCONNECTED MESH Few interconnection switches |
| IOT “Internet of Things”, interconnection via the internet of computing devices enabling them to send and receive data |
| MACHINE LEARNING Element of artificial intelligence. Computer algorithms that improve automatically through experience and by the use of data |
| MACs “Moves, Adds and Changes” |
| MDA “Main Distribution Area” |
| POWER CABLE OR POWER CORD Primary cable that provides power to the computer, printer, monitor, and components within a computer |
| PUE “Power Usage Effectiveness”, ratio that describes how efficiently a computer data center uses energy; specifically, how much energy is used by the computing equipment |
| ROUTING PLAN Planned path between networks |
| SERVER AIRFLOW Measured in cubic feet per minute (CFM), the volume of air moving through a server or cooling unit |
| SPINE LEAF Network structure that features several paths between connection switches and access switching |
| SWITCH FABRIC NETWORK A type of network architecture that includes full mesh, interconnected mesh, centralized and virtual switch options |
| THREE-TIER Traditional data center architecture in which servers connect to access switches, then aggregate switches and finally to core switches |
| VIRTUAL SWITCH Connects virtual machines (VM) with both virtual and physical networks |
| VM “Virtual Machine”, a software program that allows one virtual machine to communicate with another |
Download the Whitepaper
Enter your details below to download the Whitepaper PDF:
