History of Database

The history of databases is a fascinating journey. It evolved from the physical notebooks to the digital storage. Before computers, information was stored manually:

• Filing Cabinets:
  • Offices used filing cabinets to store physical records such as tax documents, health records, and family recipes. Each document was organized by categories, dates, or other attributes, ensuring that important data could be located when needed.
• Manual Retrieval:
  • Although this method worked for small amounts of data, it was labor-intensive and prone to errors. Finding a specific file could be time-consuming, and maintaining consistency across was challenging.

1️⃣ Pre-Database Era (Pre-1960s)

Long before computers existed, people were already organizing information in ways that resemble modern databases. Many of you might even recall your parents' filing cabinets filled with health records, tax documents, and cherished family recipes.

• Flat File Systems:
  Early computer systems stored data in simple flat files—plain text or binary files with a linear organization. Each file contained records without any internal structure to represent relationships between data elements.
• Sequential Files:
  Data was often stored sequentially, meaning that to access a particular record, systems would scan through the file from the beginning. While efficient for small datasets, this approach quickly became unwieldy as data volumes grew.

These methods marked the first step in automating data management, replacing manual processes with digital storage, though they lacked the sophistication needed for complex queries and data interrelationships.

• Manual Record-Keeping: Data was stored in physical files, ledgers, and punch cards.
• File-Based Systems (1950s - 1960s):
  • Early computers used flat-file systems, where data was stored in unstructured text files.
  • Drawbacks: Data redundancy, inconsistency, and lack of relationships between files.
  • Example: Magnetic tapes for sequential data access.

2️⃣ Birth of Database Management Systems (1960s)

• Hierarchical and Network Models:
  • Hierarchical Model: IBM’s IMS (Information Management System) in 1966, designed for NASA’s Apollo program. Data was organized in tree-like structures (parent-child relationships). Each record was linked in a parent-child relationship, making it easier to access related data. However, this structure was rigid and struggled to represent more complex relationships.
  • Network Model: Charles Bachman’s Integrated Data Store (IDS) in 1963, using graph-like structures for many-to-many relationships.
• Drawbacks: Complexity in querying and rigid structure.

3️⃣ Relational Database Revolution (1970s)

In 1970, a researcher at IBM published a paper:

"A Relational Model of data for Large Shared Data Bank"

Author: Edgar F. Codd

• Edgar F. Codd’s Relational Model (1970):
  • In 1970, Edgar F. Codd proposed a new way to structure data by using tables to represent relationships. This revolutionary idea formed the basis of relational databases, which organized data into rows and columns, making it both easy to understand and manipulate.
• Standardization and SQL (Structured Query Language):
  • Developed by IBM (Donald D. Chamberlin, Raymond Boyce) in the 1970s as SEQUEL.
  • Became the standard language for relational databases.
• Early Relational Databases:
  • System R (IBM, 1974) and Ingres (UC Berkeley, 1974).

Codd's Revolutionary Idea

Instead of:

Trees
Graphs
Pointers
Navigation

Represent data as:

Relations

which we now call:

Tables

Example:

Employees:

No pointers.

No tree.

No graph navigation.

Just relations.

The Mathematical Foundation

Codd based his model on:

Set Theory
Predicate Logic
Relational Algebra

This was revolutionary.

For the first time:

Database Design

became mathematically rigorous.

The Biggest Innovation

Old Systems:

Tell database HOW

Example:

Go to Department
Find Employee
Move to Next Employee

Relational Systems:

SELECT * FROM employees WHERE department='Engineering';

You only specify:

WHAT

not:

HOW

Database decides how.

This is called: Declarative Querying

The NoSQL Movement – Around 2005-2010

Developers realized:

Not every problem needs relational databases.

“NoSQL” emerged.

Meaning:

Not Only SQL

Companies faced:

Massive Scale
Huge Data Volume
Global Distribution

Traditional databases struggled with:

Petabytes of data
Thousands of Servers
Global Replication

This led to NoSQL.

Examples:

• MongoDB
• Apache Cassandra
• Redis

Did SQL Die?

No.

Quite the opposite.

Today:

SQL + NoSQL

Most systems use both.

Example:

PostgreSQL → Source of Truth

Redis → Cache

Elasticsearch → Search

Kafka → Event Stream

Each solves different problems.