New Tech Tuesdays: Big Data, Bigger Insights: The Power of Datafication

A May 2022 annual report titled Recent Trends in U.S. Services Trade by the U.S. International Trade Commission (USITC) stated that, “from 2010 to 2020, the volume of data that the world generated and replicated increased from 2 zettabytes (ZB) to 64.2ZB” and that “as of 2018, over 90% of the data that existed worldwide had been generated in the previous two years alone…”

The digital revolution and the information age have led to an unprecedented generation and storage of data. Big data experienced a surge in the 2000s as e-commerce, Internet of Things (IoT), and social media became ubiquitous. Businesses recognised the potential value of data. Combining big data and cloud computing enabled businesses to store and process vast amounts of data at a relatively low cost, making it more accessible and affordable. Finally, in the last few years, rapid advances in artificial intelligence (AI) and machine learning (ML), a subset of AI, have made it possible to extract valuable insights from massive amounts of data quickly and accurately. Like the perfect storm, big data, cloud storage, and AI/ML spawned a new modern and central business strategy trend known as datafication.

Datafication: From Physical Objects to Digital Data

Datafication refers to the process of converting various forms of information, such as human behavior, physical objects, or business processes, into digital data that can be analysed and used for various purposes. It involves the use of technology to collect, store, and analyse vast amounts of data, transforming them into valuable insights that can inform decision-making and improve performance.

The term datafication is often associated with the rise of big data, ML, and AI, which have enabled organisations to capture, store, and process enormous amounts of data from various sources, including social media, mobile devices, sensors, and the IoT. Datafication enables businesses to gain deeper insights into consumer behavior, optimise operations, and develop new products and services. But it has also raised concerns about security, privacy, and the potential misuse of data.

Data Patterns Are Unleashing Valuable Insights

As more people, processes, and devices become connected and interconnected, patterns in big data begin to emerge, offering new insights into how we live, work, and interact with the world around us. However, for many years much of this data has remained siloed and inaccessible, stored in remote servers and hard disks that makes up the vast "cloud." But, with the rise of AI and ML, companies and organisations are now able to extract valuable insights from this stored data and are beginning to unlock their full potential.

Data Storage Is Big Business

Most of this big data is housed-stored in massive data centers. Data centers house computers and networking equipment. This equipment includes vast server storage space that makes up the cloud along with a whole host of other equipment. Data centers also include high-speed fibre optic cable connections to the internet and other networks for transferring large amounts of data. The operations staff helps monitor the data center’s operations and maintain the information technology (IT) equipment and infrastructure.

Data centers can be privately owned—for example, those from Google and Meta—or colocation leased. A colocation data center is a type of data center where equipment, space, and bandwidth are available for rental to retail customers. In addition to providing data space, power, cooling, and security, colocation data centers interconnect a variety of telecommunications and network service providers with a minimum of cost and complexity to other firms' servers, storage, and networking equipment. Specialist companies like Equinix and Digital Realty Trust are prime examples of colocation leasing companies.

According to the same May 2022 USITC report, in 2020, the colocation segment of the data center market was valued at $54 billion, with the United States (US) having the most data centers, followed by Germany, the United Kingdom, China, and Canada. In 2020, the US had 611.8 megawatts (MW) of data center capacity under construction, which increased to 680.8MW in the first half of 2021, with Northern Virginia as the largest market for new construction.

According to the 2021 Data Center Real Estate Review published by the North American Data Centers, in 2017, there were six wholesale leases of at least 10MW and none over 25MW; in 2020, there were six wholesale leases over 40MW and 17 that were 10MW or over. In 2021, there were 11 leases over 30MW. The report also states that to accommodate these mega-leases, data center developments will need to be much larger in the future.

Pros and Cons of Datafication

Datafication has both advantages and disadvantages. On the one hand, datafication creates a data-driven culture, provides insights into customer behavior, and increases efficiency. On the flip side, datafication can lead to data privacy concerns and be a source of bias and error.

Pros:

• Improved decision-making: Datafication can help organisations make better decisions by giving them insights into customer behaviour, market trends, and business operations.
• Increased efficiency: Datafication can help automate many processes, which can increase efficiency and reduce costs.
• Personalisation: Datafication can enable personalised marketing, tailored products and services, and customised experiences for customers.
• Innovation: Datafication can lead to the development of new products and services, as well as new business models.

Cons:

• Privacy concerns: Datafication raises concerns about privacy, as personal data can be collected and used without the knowledge or consent of individuals.
• Security risks: Datafication also raises concerns about security, as the more data that is collected and stored, the more potential there is for breaches and cyber-attacks.
• Bias and discrimination: Datafication can perpetuate existing biases and discrimination, as algorithms may be trained on biased data, leading to unfair outcomes.
• Overreliance on data: Datafication can lead to an overreliance on data and a lack of intuition and creativity in decision-making.
• Digital divide: Datafication can widen the digital divide between those who have access to technology and data and those who do not, leading to further inequality.

Examples Where Datafication Is Making a Difference

Datafication is being used across a wide range of industries and applications, from healthcare to finance to retail. Here are a few examples:

• Healthcare: Datafication is being used in healthcare to improve patient outcomes and reduce costs. One example is the use of electronic health records (EHRs), which enable healthcare providers to collect and analyse patient data more efficiently.
• Finance: Datafication is also being used in finance to improve risk management and fraud detection. For example, banks and credit card companies are using machine learning algorithms to detect fraudulent transactions in real time.
• Retail: Datafication is being used in retail to improve customer engagement and personalise the shopping experience. For example, online retailers, such as Amazon, use customer data to recommend products and personalise marketing messages.
• Transportation: Datafication is also being used in transportation to improve safety and efficiency. For example, self-driving cars use sensors and machine learning algorithms to navigate roads and avoid accidents.

Featured Products

This week’s New Tech Tuesdays introduces two new products from Micron and Xilinx that help enable the concept of datafication to transform industries and drive innovation across the globe.

Micron 9400 NVMe® SSDs

The 9400 NVMe SSDs are the ultimate solution for performance-critical data center storage. With 30TB of usable capacity, they outperform competitors by 2.3x in mixed workloads and improve power efficiency by 77%. Other key benefits include:

• Optimises critical workloads: caching, content delivery, block/object stores, and Al training/caching.
• Achieves industry-leading sequential read/write speeds of 7GB/s, surpassing competitors by 66% in sequential writes.
• Offers the industry's fastest random read/write performance, reaching up to 1.6M IOPS.
• Provides uncompromising data center performance in a compact package.
   

The Micron 9400 SSDs unleash unparalleled performance for critical data center workloads with high capacities and exceptional efficiency.

Xilinx Alveo™ U250 Data Center Accelerator Card

The U200 and U250 Data Center accelerator cards are high-performance PCIe Gen3 x16 compliant cards designed for compute-intensive applications like machine learning, data analytics, and video processing. These cards are built with custom UltraScale+™ FPGAs optimised for the Alveo architecture and employ Xilinx’s stacked silicon interconnect (SSI) technology, enabling breakthrough FPGA capacity, bandwidth, and power efficiency. Both cards connect to 16 lanes of PCI Express, supporting speeds of up to 8GT/s (Gen3). They also connect to four DDR4 DIMMs with ECC, providing a total of 64GB of DDR4 memory. Target applications will benefit from these cards enabling solutions that can be deployed interchangeably in the cloud or on-premises, scaling to meet specific application requirements.

Tuesday’s Takeaway

Datafication has emerged as a powerful force across multiple industries, enabling organisations to collect, process, and analyse vast amounts of data to drive insights and value. While datafication has the potential to transform organisations and drive innovation, it also presents challenges around data privacy, security, and bias. To leverage the full potential of datafication, organisations must build a strong data culture, invest in robust data governance frameworks, and embrace emerging security technologies to enable secure and transparent data sharing. As datafication continues to evolve, it will be critical for organisations to stay ahead of the curve and embrace new tools and techniques to drive insights and value from their data.

Image credit: iStock.com/everythingpossible