{"id":73133,"date":"2024-05-06T08:00:47","date_gmt":"2024-05-06T15:00:47","guid":{"rendered":"https:\/\/phisonblog.com\/?p=73133"},"modified":"2025-07-22T09:21:42","modified_gmt":"2025-07-22T16:21:42","slug":"reducing-data-volume-the-value-of-deduplication","status":"publish","type":"post","link":"https:\/\/phisonblog.com\/de\/reducing-data-volume-the-value-of-deduplication\/","title":{"rendered":"Reduzierung des Datenvolumens: Der Nutzen der Deduplizierung"},"content":{"rendered":"

[et_pb_section fb_built=”1″ _builder_version=”4.16″ _module_preset=”default” custom_margin=”0px||||false|false” custom_padding=”0px||||false|false” locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.16″ _module_preset=”default” width=”100%” max_width=”100%” custom_margin=”||||false|false” custom_padding=”0px||||false|false” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.24.3″ _module_preset=”default” header_2_line_height=”1.7em” header_3_line_height=”1.7em” custom_margin=”||-10px||false|false” custom_padding=”||0px||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

The world is experiencing an explosion of data like never before and organizations must find new, more efficient ways to store, manage, secure, access, and use that data. A lot of valuable insights lie hidden within the types of data being generated today, and those insights can help organizations identify production bottlenecks, improve the customer experience, streamline processes to increase agility and much more.<\/p>\n

At the same time that data volumes are skyrocketing, the costs of storage infrastructure and management tools are diminishing. These factors often drive organizations to embrace the strategy of storing all of their data for long periods of time\u2014or forever\u2014no matter what it is or where it came from.<\/p>\n

Just because you can store more data more cheaply today, it doesn\u2019t necessarily mean you should do so indiscriminately. Not all data is created equal, and some types of information contain much more value than others.<\/p>\n

There can also be a lot of redundancy in data stores. If you have information pouring in from your customer relationship management platform, sales, technical support, human resources, product marketing and so on, there can be overlap. Duplicate data can also be generated through regular backups, file sharing, data entry or import\/export errors, inaccurate data input by customers and so on.<\/p>\n

This redundancy can bloat your stored data volumes and make it harder to pinpoint the information you need in the moment you need it. In addition, it can drive up storage costs. While storage is cheaper now than it was before, there\u2019s still no reason to pay for more than you really need.<\/p>\n

Data reduction techniques<\/a> allow organizations to reduce the overall size of their data, which reduces their storage footprints and costs and improves storage performance. One of the valuable tools in the data reduction toolkit is deduplication.<\/p>\n

 <\/p>\n

\"\"<\/a>
Read: How Hyperscalers Can Maximize Data Storage Capabilities<\/div><\/div><\/div><\/div><\/div>\n

 <\/p>\n

What is data deduplication and how does it work?<\/h3>\n

Data deduplication is a type of data compression that deletes redundant information on a file or subfile level. In a large global enterprise, for instance, that redundant data can take up a lot of space in the company\u2019s storage systems. By eliminating duplicate information, that enterprise\u2019s systems will retain just one copy of that data.<\/p>\n

To dedupe data, an application or service will analyze entire datasets at the level of files or blocks. It is often done in combination with other data compression techniques to significantly reduce data size without compromising its accuracy and authenticity.<\/p>\n

File-level data deduplication was the first type of dedupe and it involved deleting redundant copies of files. In place of those deleted files, the system would create a sort of digital \u201cpointer\u201d that would point to the original, retained file in the repository.<\/p>\n

File-level dedupe is a bit limiting, however. Consider how people share documents today and make changes and updates. Different versions of the same document, containing minor differences only, weren\u2019t considered duplicate.<\/p>\n

Block-level data deduplication is more granular. It goes deeper into the data and is therefore more effective at rooting out duplicated data within a file. It works by assigning a \u201chash\u201d to each block of data\u2014blocks being smaller chunks of information within a file\u2014and that hash acts as a unique identifier or signature of the block. If the system detects two identical hashes, one is deleted as duplicate.<\/p>\n

So, for a document file that has been changed, instead of saving the entire document again with minor changes, the system will only save the blocks that have changed in the new document\u2014retaining the original as well as the minor changes.<\/p>\n

Depending on the system, there are two approaches to data deduplication:<\/p>\n