IT Tips & Tricks
Published 31 October 2017
Updated 7 September 2021
On Moving Your Data to the Cloud: “Resistance is Futile” and Unnecessary
Editor’s Note: This article was originally published on Halloween of 2017, yet it only took a few edits and updates to make it as fully applicable as ever. We also added a useful tip on how to prevent missing data due to the broken file links that accompany most data migrations (particularly to the cloud).
Like the gravity of Jupiter, the pressure on corporations, non-profits and governments to move data and computing resources to the cloud has become massive. For many organizations, it’s a no brainer. The cloud is a godsend. For others, the cloud can seem to solve nothing and possibly create problems of its own. Most organizations find themselves somewhere between these two extremes. For some perspective on all this, let’s look at the how this migration juggernaut is influencing certain financial, security and staffing decisions in various businesses. Then we’ll look at how it affects you.
The Cloud is a lot like the cardboard collar on a hot coffee cup. Once a novel idea, it’s now become a practical necessity: something that simply makes our lives better.
Ah, the 80’s. Young people sold computers from their parents’ home or their dorm room. That’s the way Michael Dell started. While many consultants still sell hardware to their clients, today, they could just spin up a virtual server.
Financial Impact
Sometimes Free drives the move.
A virtual computer (“the cloud”) is less expensive than a real computer. That is, if two small IT consulting shops had a new client order a server, the financial and time costs would be quite different. One could spin up a server in less time than the other shop could get the computer ordered. Even with Amazon Prime, the shop ordering a physical server would need to wait two days. While the other shop would have the server up in an hour or so. (Score one for the cloud.)
Even if the shop that was sourcing hardware drove to the local, big box store, bought the computer and drove it to a local client, the virtual install would still be faster. And that virtual install would not require the need to leave the office. Plus, one could set-up a cloud server with the client’s billing info and corporate credit card. All without having to buy and hold inventory.
With more and more businesses and governments realizing that setting up virtual servers makes business sense, the economy of scale is driving cloud costs down. The proof comes when you look at blade servers versus standalone PC sales. Blade servers are quickly outpacing PC sales. Another sign is that lots of business are using the likes of Amazon Web Services (AWS), Microsoft Azure and Google Cloud (to name just the largest ones). [Editor: Interesting that it’s been four years since the first iteration of this article and yet the same providers are still the “big three”. Although, in fairness, we should add IBM Cloud to the “big” list. (Of course, bigger does not necessarily mean better; but that’s a topic for an entirely different article.)]
Sometimes Free drives the move. Given that the big three cloud providers offer some free time to new users, many small businesses are putting their websites on cloud servers instead of buying hosting packages. All this is driving down the cost of cloud computing — which means owning on-site hardware is quickly becoming more expensive, relatively speaking. If your competitor has moved to the cloud, they may enjoy a cost competitive advantage. And this is putting pressure on CIOs to deliver financial results or justify why they are keeping hardware in house.
If your competitor has moved to the cloud, they may enjoy a cost competitive advantage.
Staff Costs in Your Data Center
With companies like Rackspace.com specializing in — well — rack servers, it is becoming hard to justify the salaries of IT staff for private corporate data centers. Rackspace gets more mileage out of its staff (dedicated to running their data centers) because their business is about being, “your data center” (as is the staff at AWS, Azure, Google Cloud and the like).
This trend means CIOs are facing pressure to close or consolidate their data centers. And that can mean taking advantage of lower-cost real estate. Why have a center in New York, when you can have one in Quincy, Washington (population 6,000) where Dell, Yahoo, Microsoft and other big players have data centers? The cost savings can be huge. Staff, electricity and office space are cheaper.
This trend means CIOs are facing pressure to close or consolidate their data centers.
And as more businesses become data driven, the cost savings just keep adding up. But it’s not just cost. Closing data centers allows a business to focus on its core competencies. Do you really need the CIO and staff to run a data center when other IT skills are more important to customer retention, acquisition and satisfaction?
Running your own data center can be a distraction.
If it’s not part of your core mission, maybe it should be outsourced. Often the main customer data is quite small. Think of eBay. Lots of users, yes, but do they really need to keep all the auction data years after the sale? Really, the customers are an asset, but once you have recorded the statistical data, old auctions are likely not.
eBay may prefer to keep their data in house, but they don’t need to.
And that is becoming the crux of the problem. Center providers are becoming so good, efficient, and cheap, that corporations need to make a hard decision. What do we really need to have on our servers, and what should we outsource? Another advantage of remote data centers is colocation with customers. With AWS, for example, one can split data between West Coast and Eastern centers, thereby insuring that data is served with as little latency as realistically possible. Having data close to the customer can do that. And it can be done seasonally (as an example). What do we mean by this?
A lot of financially independent people are called “Snowbirds” because they live in New York (for example) in the warm season, but once the snow flies, they’re off to sunny Florida (or Southern California). With a fixed data center, the distance to the customer base is fixed (duh). Now take that same data, and put it on a cloud service with data centers on the upper East Coast, and one on the lower East coast. Run the servers where the clients are and reduce latency. In the warm months, spin up more of the northern-most servers and shut down a lot of the southern ones. In the Winter, reverse it. If you did this with real, physical office space, you’d pay rent even when not using the space to capacity. With remote data centers, you can spin up, or down, servers in one region — and do the opposite elsewhere.
Have you ever tried to restore a terabyte drive from the cloud? A lot of small business don’t understand that cloud backups are not the panacea they are advertised to be.
This allows businesses with seasonal customers or corporations with migratory clients to have the data near the client without renting more space. Honey Baked Ham is an interesting example. As you can expect, they do a crushing amount of their business during the holidays. But orders come in months before. Here is a perfect business that needs to add capacity seasonally. Rent is a fixed cost per year. But from the viewpoint of a business, remote centers act like a variable cost, allowing the organization to adjust the data resources as needed. Having office space and servers that lay idle much of the year is hard to justify. Gone are the days when Henry Ford wanted to own every plant, that made every part, of what went into a car. Today, organizations need to think what centers they really need to own.
Security
What you own, you also need to protect.
This makes IT staff question what should be kept in corporate locations and what should be backed up to the cloud.
Have you ever tried to restore a terabyte drive from the cloud? A lot of small business don’t understand that cloud backups are not the panacea they are advertised to be. Some remember the days of tape drives. IT staff would tell clients to test the backup now and then to make sure the drive worked. And even test the backup on a second computer to make sure that any drive misalignment would be detected and the drive replaced.
The number of small business that are blindly using cloud backups is driving that industry to lower and lower prices. But also, these providers are becoming really good at verifying, cloning and protecting data. While Dropbox is not encrypted, most cloud backup services offer end-to-end encryption. And again, we see economy of scale at work exponentially. That is to say, more companies utilize the service because the price is reduced — and the price is reduced by more companies using it.
Online backup companies offer secure physical sites (often not publishing where their data centers are), redundant backups, encryption, and other measures to keep their clients’ data secure. This makes IT staff question what should be kept in corporate locations and what should be backed up to the cloud.
Local storage is vulnerable to physical attacks, fire, accidents, and weather events. Offsite data is stored generally in multiple, hardened locations. Corporations may worry about disgruntled employees, angry customers or public outraged by some corporate policy, any of whom could seek revenge by deleting data. Keeping data offsite, so even the company’s own staff do not know its location, helps alleviate this concern.
But that’s not the only advantage of offsite storage. With local storage, staff often know what kind of sensitive information is in the data, such as Personally Identifiable Information (PII). A person working at a credit card company knows that the data files contain social security numbers. But workers at cloud data centers, do not know what’s on the encrypted drives. With so many clients, an employee at a cloud facility would have a hard time knowing what data is being stored by which client. Additionally, these facilities prohibit employees from walking around with flash drives, and implement other measures to assure that data is not being accessed improperly. Everyone at a cloud center knows that it’s not their data.
Admittedly, there are dozens of factors to consider when choosing whether to move to the Cloud or not and in determining which provider is right for you, but the benefits are worth it.
Often, offsite data centers are staffed 24/7. Through economies of scale, and since cloud storage is their primary business, cloud companies should have a lower IT cost.
Storing data offsite does have some downsides. Large data sets like video records can clog a network. Most movie editing companies have local video servers, even if they use offsite backup. And for 3D rendering, animation studios depend on local storage. Cloud-based video editing is a non-starter — it’s just too slow.
Companies that depend on a huge number of small transactions, often need local servers to optimize their transaction speed.
The point is, that companies need to look at speed of access, security and the cost to maintain data when deciding between local and offsite. Cost is key here. With physical products, a warehouse calculates the cost per square foot. This can be added back to the project as a “storage cost.”
With data, there is some cost to hold that data (cost per storage GB) but the real cost is staff to maintain the data. Maintaining means the cost to backup, the access speed for the users, and the security cost to protect it. All of which become a cost of the data.
Now we’re entering the era when buying hardware to host IT in-house may not make sense.
With ransomware and frightening Social Security leaks such as at Equifax and Sony, one needs to ask what should be backed up to more secure servers. The Sony breach showed Social Security numbers for employees that left the company two years prior to the breach. Why not have that data archived on the cloud? With the Equifax breach, we may see lawsuits that start to cause corporations to rethink having all their data in house and online all the time. Secure backup may be a better solution for older data, backups of key data, and data that should be accessed only for a single transaction.
This CIO.com article offers some additional advice on security in the cloud.
IT staff in the crosshairs
It’s a numbers game. The volume of smaller-demand users is enough to drive prices down while demanding better services.
As strange as it may seem, it’s the lowest demanding users that are driving corporations to rethink in-house IT resources. Individuals buying cloud backup services for wedding photos, small business owners running WordPress sites on free cloud servers, and consultants spinning up virtual machines are putting pressure on corporations to follow suit.
It’s not that these smaller-demand users are buying a lot, but there are a lot of them buying. And the massive buying power is driving cost down, service offerings up, and making the case to use more in the C-suite.
A lot of small business owners buy hosting services for a WordPress site (their small business site). Most of these sites get almost no traffic. Yet they pay each year. So hosting companies know they can get a lot of small businesses to pay for services they don’t fully use. That excess capacity is why hosting is so cheap.
How many small businesses are really taking advantage of the free cloud services offered by the big three (AWS, Azure, Google)? It’s not mainstream yet, but it’s growing, and giving traditional hosting companies a run for their money. As cloud-computing pricing races to the bottom, CIOs have a question to answer: How little hardware do we really need?
Large factories, like an automobile plant, would evaluate the investment in building their own power plant versus buying power from the local utility. Today, many IT departments are in a similar situation — evaluating the costs to keep data in onsite hardware versus offsite storage services. Now we’re entering the era when buying hardware to host IT in-house may not make sense. Or at least it should be weighed.
Yet all these cost savings do have a dark side . . .
Cost to Migrate
Yet all these cost savings do have a dark side. For many corporations, migrating to the cloud can be challenging. For example, whenever files or folders are migrated, links get broken, sometimes by the thousands or millions. This results in data missing from files which comes with an obvious time and money drain. So these links have to be fixed, but this is a slow, man-hour-intensive process even if you do use a search & replace script. [Editor: Fortunately, there is a solution for this one and it comes down to preventing the problem in the first place. Visit www.LinkTek.com. This tool could save you hundreds or even thousands of IT man-hours, in addition to saving users from downtime which is potentially even more costly.]
This tool could save you hundreds or even thousands of IT man-hours, in addition to saving users from downtime which is potentially even more costly.
Some corporations are moving to “online access” of data as the only option. No more local storage. This is simpler with Apple tablets where local storage can be locked down. This eliminates the worry of flash drive data leakage.
With mobile phones providing a persistent connection, and cloud software like Microsoft Office 365, the concept of moving a file to your local device to edit it is antiquated. The benefits of Microsoft’s cloud-centric solutions, especially Azure, are not insignificant, as outlined in this iCorp blog article.
Once an organization moves to cloud only, or some subset of that, the savings may not come all at once. This is because the move may take years until an organization reaches the right balance. For the next decade, expect to see the move continue, as any smart CIO will rebalance internal and external resources. If you work for one, be on the lookout for tools to help with migration and the right suite of services. Using AWS, Azure and Google Cloud took me quite some time to learn. Each has a swirling bevy of options. But the benefits are making the case for migration to the cloud irresistible — which also means moving away from the file-based infrastructure of the past and moving toward a cloud-centric model of data usage.
Feel free to share this article on your social media:
ChatGPT,
Gemini,
Claude and
Copilot