Written by Data lakes typically store a large amount of raw data in a single place. They are becoming more and more popular for businesses because they allow extensive data exploration.
Contents
What is a Data Lake?
A data lake is a place for storing both structured and unstructured data. It’s a centralized repository designed not only to store data but also to explore, analyze and secure large volumes of data from various sources.
Data lakes contain really vast amounts of data (they can reach sizes up to petabytes) therefore they are usually kept in cloud-based storage. Examples of such cloud storage are Azure, Google Cloud Storage, AWS S3, Wasabi, etc.
What Type of Data can you Store Here?
In data lakes you can store various formats of data:
- Unstructured data
- Semi-structured data
- Structured data
Unstructured Data
Unstructured native data is anything that doesn’t have a specific format. Unstructured data include text, images, location data, log data from servers, social media comments and posts,…
Semi-structured Data
Semi-structured data is partly structured and has some consistent characteristics. It has some properties such as metadata semantics tags, internal tags, and other marks that help to identify groups and hierarchies. Examples of semi-structured data are emails, hierarchical web content, XMLs, NoSQL databases, and more.
Structured Data
Structured data has been formatted and transformed. Its elements are structured into fixed pre-defined fields. Examples of structured data include databases consisting of tables with rigidly structured rows and columns. Other examples include barcodes, web statistics, addresses, demographic information, accounting transactions, etc.
Features
Data lakes are different from other types of data storage concepts.
These are the main characteristics that distinguish them:
- You can store here any type of data from different sources
- Data is stored in its native format without transformations (raw state)
- You can transform data for analysis anytime (based on search criteria)
How do Data Get into Data Lakes?
Professionals, such as data analysts or business managers, firstly identify interesting sources of data. If they find the data important, they replicate it to the data lake (usually without any modifications). These raw data are then available for further analysis or machine learning.
Businesses nowadays have really huge amounts of data from diverse sources. There’s no wonder they want to make use of it to achieve their business goals. One common goal among all businesses is to find correlations between different data sets and thanks to combining them improve customer experience.
All data in a data lake is available on-demand, so companies can use it according to their needs. When they want to analyze a data lake it provides them with a subset of data based on matching query criteria.
Advantages of Data Lakes
Some of the benefits of data lakes include:
- Versatility – ability to store various forms of data (structured/unstructured data) and also ability to make use of these data.
- Flexibility – data analysts can easily organize and analyze data according to their queries.
- Complexity – elimination of data silos by combining data from all of the sources.
- Accessibility – data are available to the whole organization (this is also called democratization).
- Scalability – capability of a data lake to manage a growing volume of data.
- Advanced Analytics – data lakes have ability to use large amounts of data along with deep learning algorithms. It can help in real-time decision analytics. This is also a difference between data warehouses and lakes.
Data Lake vs. Data Warehouse
What is the difference between data lakes vs. data warehouses?
They are both big data storage but they serve different purposes.
A data lake contains a large amount of unstructured data. On the other hand, a data warehouse stores structured and filtered data that has been modified for a specific purpose.
Another notable difference between these repositories is that a data lake doesn’t have a predetermined schema while a data warehouse stores data in a predetermined organization with a schema.
These are some of the other differences:
| Characteristics | Data Lake | Data Warehouse |
| Data Format | Unstructured and semi-structured | Structured |
| Purpose of Data | Doesn’t have a determined purpose | Has a specific purpose |
| Schema | Schema-on-read: doesn’t have predetermined schema | Schema-on-write: predetermined |
| Users | Data scientists, Data developers, and Business analysts | Business analysts |
| Scalability | Highly scalable: hold any amount of data of any type | Scaling is more expensive |