Blog

We frequently hear from our readers that some of our best cloud storage services, especially Dropbox and OneDrive, sync files much faster than their competitors like Google Drive, Box and pCloud. There’s a reason for that, and it’s called block-level sync — sometimes known as “differential sync,” “delta sync” or “incremental backup.”

Key Takeaways:

• Block-level sync means that a service copies only the parts of a file that have changed, rather than the entire file. It makes for faster syncing between devices, but has little to no impact on initial upload and download speeds.
• There’s no conflict between block-level sync and zero-knowledge encryption. A service can have both.
• Block-level sync is also called delta sync or differential sync. The latter should not be confused with differential backup, which refers to backing up select files from a whole device or drive.

Sync speeds are a big deal, particularly if you use cloud storage to collaborate with your coworkers. The faster your sync speeds, the closer you can get to sharing real-time updates. Even if you just use cloud storage for personal files, faster syncing lets you upload files in a hurry and access them from other devices sooner.

Although block-level sync is faster than its alternatives, it’s still mainly used by Dropbox and OneDrive, which appear to have perfected the technology in a way competitors struggle to copy.

In this article, we’ll discuss the basic mechanics behind block-level copying, then talk about some of the cloud storage and online backup services that use this technique. We’ll also go into why some services don’t use it, and what this all means for you.

• Does Google Drive Use Block-Level Sync?

  Google Drive does not support block-level file sync, in spite of some sources suggesting it does.

• Is OneDrive Block-Level Sync?

  Microsoft OneDrive applies block-level file sync to all files. Before 2020, it was only available for Microsoft file formats, but it’s now open to all.

• What Is Block-Level Transfer?

  Block-level transfer means breaking up files into blocks before moving them between systems. It’s the process that underlies block-level file sync.

Google Drive does not support block-level file sync, in spite of some sources suggesting it does.n”}},{“@type”:”Question”,”name”:”Is OneDrive Block-Level Sync?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”

Microsoft OneDrive applies block-level file sync to all files. Before 2020, it was only available for Microsoft file formats, but itu2019s now open to all.n”}},{“@type”:”Question”,”name”:”What Is Block-Level Transfer?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”

Block-level transfer means breaking up files into blocks before moving them between systems. Itu2019s the process that underlies block-level file sync.n”}}]}]]>

Basics of Block-Level Sync

Block-level file copying is a pretty simple concept. An online storage service “syncs” files by saving copies of them on its own servers. When you make a change to the file, a service with block-level copying will only copy the parts of the file that changed, instead of copying the whole file again.

It manages this by breaking each file into blocks. For example, Dropbox divides every file on its servers into blocks with 4MB of data each. These blocks are hashed with SHA-256 encryption, then Dropbox stores a list of the hashed blocks for reference.

By only uploading the parts of a file that has changed, transfer times can be reduced to fractions of what it would be to update the entire file.

When syncing each file, Dropbox checks to see if any of the hashed blocks recorded on the blocklist are missing. If a block in the file matches a block on the list, it doesn’t need to be synced. If it doesn’t match, Dropbox returns instructions to upload that block, since it contains new content.

Note that the cloud storage service doesn’t need to decrypt the file to find the changed blocks. It can just compare the hashes. This will be important when we talk about zero-knowledge encryption.

Of course, if the file isn’t yet stored on the servers, Dropbox (or the other block-level storage service) will have to upload the entire file. This is why the initial upload of any file on Dropbox takes longer than subsequent syncs. It’s also the main area in which you’ll see competitors claiming a speed advantage.

Cloud Services That Use Block-Level Copying

Given that differential sync appears to be more efficient, especially for large databases, you might assume every cloud storage and backup service would use it. However, among cloud sync software, it’s not as common as you’d think. Only about a third of top services support block-level file sync, including Dropbox, OneDrive, pCloud and Amazon Drive.

The much longer list of well-known services that don’t support differential copying includes Google Drive, Box, Icedrive, Koofr, Tresorit, Egnyte, MEGA and iCloud. (Note that we found some conflicting information about whether Google Drive supports block-level file sync, but ultimately discovered no evidence that it does so).

Why wouldn’t a cloud storage platform use block-level sync, especially for large files? Mainly, it’s because block-level sync doesn’t make a difference when uploading or downloading new files — it only helps when you need to upload changes quickly, usually to make sure multiple devices all hold the same version of a single file.

Block-Level File Sync and Speeds

For our recent article on the fastest cloud storage services, we tested 17 services to see how quickly they could upload and download a 5GB folder containing many different file types. We uploaded and downloaded the folder twice for each service, then took the average to get our final result.

The outcome for the upload time test was close. Less than a minute separated the top five finishers, and just over two minutes separated the top 10. The table below holds the results, measured in hours, minutes and seconds.

Upload

* = Zero-knowledge encryption enabled

Icedrive came in first, taking an average of 7 minutes and 15 seconds to upload 5GB. You may recall from the last section that Icedrive doesn’t use block-level file sync. Neither does our second-fastest contestant, Google Drive, which took 7 minutes and 21 seconds on average. It’s the same story with third-place Koofr (average of 7 minutes, 42 seconds).

Download speed was a different story, and provides additional evidence for why not every cloud storage developer flocks to block-level sync. Take a look at our results below.

Download

The services deliver a much more consistent showing here, with all the contenders clocking in less than 10 minutes apart. The top four services with block-level file sync (pCloud, Amazon Drive, OneDrive and Dropbox) are all in the top seven, but three file-level services (Icedrive, Google Drive and Tresorit) match them.

Why Doesn’t Everyone Use Block-Level Sync?

Dropbox is a good case study. It has the fastest upload speed of any service with block-level sync, and the fourth-fastest overall. While it has the seventh-fastest download speed, it’s only a minute behind the leader on average. That proves it has a well-built infrastructure.

However, that’s got nothing to do with its sync methods. Differential sync only matters when working with really large files, or if a file changed on one device needs to match those changes on all the others. These are common use cases of cloud storage, but they’re far from the only ones.

Software companies have limited resources, and have to decide what to focus on. Internal market research may have shown that most users store and retrieve data without editing it in the cloud. If a block-level file sync doesn’t deliver better speeds across the board, it’s easy to understand why some teams decided to focus on something else.

Backup vs Sync: Decoding the Confusion

We’d like to detour briefly to clear up a common source of confusion. So far, we’ve been talking about using cloud storage for file sync, not system backup.

We bring this up because both cloud storage and online backup have a procedure called “differential,” and a lot of the sources we’ve examined get these confused. We’d like to set the record straight.

IDrive, our favorite cloud backup service, is about protecting your data from accidents. Cloud storage is more concerned with granting you additional storage space.

Users have three main options when syncing to a cloud backup service: full, differential and incremental. A full backup saves all your data, no matter what it is or when it last changed. A differential backup saves only the files that changed since your last full backup. An incremental backup is much faster, since it only saves the files changed since the last backup of any type.

In all three cases, we’re still saving the complete file, not just changed blocks like with Dropbox and the others. It works on the same principle — no need to recopy things that haven’t changed — but with a different scale and purpose. Block-level file sync is the sole process that saves only parts of a file to another device.

Block-Level Copying and Zero-Knowledge Encryption

We also need to address a persistent myth about block-level copying. You may have heard it reported that for a service to use block-level file sync, it cannot offer its users complete encryption. We’re happy to report that’s not true.

The myth goes like this: Zero-knowledge encryption is the most secure encryption method, in which the company can’t decode the files it stores. However, since the cloud storage service needs to know exactly what parts of each file have been changed, it must be able to read the unencrypted file. Therefore, differential sync is incompatible with zero-knowledge protection.

If you know how zero-knowledge works, you’ve already spotted the problem. As we explained above, the cloud storage platform doesn’t need to read the file; all it needs to know is whether the encrypted hash has or hasn’t changed in a given block. pCloud told us outright that its block-level sync applies to any file edited on the cloud, no matter how it’s encrypted.

This myth probably originates from Dropbox, a service well-known for using block-level file sync and lacking zero-knowledge encryption. However, correlation isn’t causation. We’ll just state one more time that there is no conflict between zero-knowledge encryption and block-level file sync.

Final Thoughts: Speed or Security

From VPNs to password managers to cloud storage, services that handle sensitive information are often portrayed as facing a choice between security and speed. However, the real world isn’t so binary. Our favorite cloud storage services, like Sync.com, pCloud and Icedrive, provide their users with both speed and security.

Block-level file sync is most important if you frequently use your cloud services for collaboration, sharing copies of files with people far away from you. If you mainly use it for storage and retrieval, this feature won’t matter as much.

We hope you now understand block-level file sync well enough to know if it matters for you when it’s being advertised. If you think we’ve missed anything important, or just want to chat about your favorite cloud services, hit up the comments section below. Thanks for reading!

Let us know if you liked the post. That’s the only way we can improve.

Yes No

Source

Originally posted on January 16, 2023 @ 11:59 am