View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

What are Data Repositories and Why Do They Exist?

Do you know a data cube from a data warehouse or a data mart?

By CBR Staff Writer

Scientific research uses data to justify reasoning and decisions. This works because data provides the information and evidence scientists need to prove that their hypotheses are correct. A similar relationship has been developing in business, with organisations analysing large data sets to help make justifiable business decisions.

This data is usually stored in a tool known as a data repository – other names include data library or data archive.

By definition data in this tool will be mined for analysis and reporting. The repository itself is an infrastructure of databases that collect, manage and store varying data sets.


What are the Different Types of Data Repository?

The term data repository can be used to describe several ways to collect and store data:

– Data warehouses are large data repositories that aggregate data from multiple sources or segments of a business, without the data being necessarily related

– Data lakes are large data repositories that store unstructured data that is classified and tagged with metadata

Content from our partners
Green for go: Transforming trade in the UK
Manufacturers are switching to personalised customer experience amid fierce competition
How many ends in end-to-end service orchestration?

– Metadata repositories store data about data and databases. The metadata explains where the data originated, how it was captured, and what it represents.

– Data cubes are lists of data with three or more dimensions stored as a table — as you may find in a spreadsheet

It is also worth noting the existence of data marts. These marts are subsets of the data repository. They are more targeted to what the user needs and are also more secure since they limit authorised users to isolated data sets. Those users cannot access all the data in the data repository.

What are the Benefits and Disadvantages of Data Repositories?

Data analysis has proven a worthy investment that can improve business decisions.  No longer are businesses making decisions based on old anecdotes and instincts. Data repositories are proving their worth in various ways:

– Isolation allows for easier and faster data reporting or analysis because the data is clustered together

– Database administrators have easier time tracking problems because data repositories are compartmentalised

– Data is preserved and archived

However, there are several vulnerabilities that exist in data repositories that enterprises must manage effectively to mitigate potential data security risks, including:

– Growing data sets could slow down systems. Therefore, making sure database management systems can scale with data growth is necessary.

– A system crash could affect all the data. Backup the databases and isolate access applications so system risk is restrained.

– Unauthorised users can access all sensitive data more easily than if it was distributed across several locations.

There will be many out there who believe storing such massive data sets in one location is quite risky. However, securing data distributed among several locations is far harder than securing a single repository, which is also far simpler to backup.

There is a valid risk here but it can be addressed with sensible data management and business-wide security policies.

What’s Best Practice for Using these Technologies?

When creating and maintaining data repositories, there are many hardware and software decisions to make. However, establishing some data warehousing best practices before this will inform the technical decisions and keep the data repository useful:

– Enlist a high-level business champion to engage all stakeholders during the project development and during its use. This is not a developer but someone who can work across departments, engaging people who will use the data repository

– The data repository will need to grow. Treat it as an ongoing system. Ensure that you have hired experts who can build and maintain the data repository as it is needed

– Don’t start off too ambitious; keep the scope of the data repository modest in early days. Collect smaller sets of data and restrict the number of data subjects. Build upon the complexity as the data users learn the system and discover return on investment

– Use Extract-Transformation-Load (ETL) tools to migrate data to the data repository. These tools ensure data quality in the transfer

– Build your data warehouse first, and then build the data marts. Decide how often the data warehouse will load new data. This often depends on the volume of data. Don’t forget that metadata is necessary for quality data analysis and reporting

– Data users need to have access to education and support

As more organisations adopt data repositories to store and manage their ever-growing volume of data, a secure approach is pertinent to an enterprise’s overall security posture.

Adopting sound security practices, such as developing comprehensive access rules to allow only authorised users with a legitimate business need to access, modify, or transmit data, is crucial. Combined with a digital signature approach or multi-factor authentication, access rules go a long way in keeping sensitive data stored in a data repository secure.

These and other security measures enable today’s enterprises to fully leverage large volumes of data without introducing unnecessary security risks.



Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.