SQL (Structured Query Language) is the standard language for storing, manipulating, and retrieving data from relational databases. If you've ever searched a website, placed an order online, or logged into an app — SQL was almost certainly running behind the scenes.
SQL is pronounced "sequel" (or spelled out as S-Q-L) and has been around since the 1970s. Despite being one of the oldest programming technologies still in use, SQL is consistently ranked as one of the most in-demand skills for developers, data analysts, and data engineers.
SQL in 30 seconds
| Concept | What it means |
|---|---|
| Database | An organised collection of structured data |
| Table | Data stored in rows and columns (like a spreadsheet) |
| Row | A single record (e.g. one customer, one order) |
| Column | A field/attribute shared by all records (e.g. email, price) |
| Query | A question you ask the database using SQL |
| Schema | The structure/blueprint that defines tables and their relationships |
| Primary key | A unique identifier for each row (e.g. id) |
| Foreign key | A column that links to another table's primary key |
Why does SQL exist?
Before databases, data was stored in flat files — CSV files, text files, or custom binary formats. Retrieving specific records meant reading the entire file and filtering it in your application code. This was slow, error-prone, and hard to keep consistent.
In 1970, Edgar F. Codd at IBM published his paper introducing the relational model, which organises data into tables with defined relationships. SQL was created as the language to interact with these relational databases.
SQL solves several fundamental problems:
- Data retrieval — find exactly the records you need without reading everything
- Data integrity — enforce rules like "every order must have a valid customer"
- Concurrent access — thousands of users can read and write safely at the same time
- Persistence — data survives application restarts and server crashes
- Relationships — link data across tables without duplicating it
How SQL works
When you write an SQL query, the database engine:
- Parses the SQL text into a parse tree
- Optimises the query (finds the fastest execution plan using indexes, statistics, etc.)
- Executes the plan — reads from disk or memory, applies filters, joins tables
- Returns the result set to your application
Your Application
│
│ SELECT * FROM orders WHERE status = 'pending'
▼
SQL Engine
┌──────────────────────────────┐
│ Parser → Planner → Executor │
└──────────────────────────────┘
│
▼
Storage Layer (tables, indexes, files on disk)
This all happens in milliseconds. A well-indexed query on a table with millions of rows can return in under 1ms.
The core SQL commands
SQL statements fall into four categories:
DDL — Data Definition Language (structure)
Create and modify the database structure.
-- Create a table
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR(255) NOT NULL UNIQUE,
name VARCHAR(100) NOT NULL,
created_at TIMESTAMP DEFAULT NOW()
);
-- Add a column
ALTER TABLE users ADD COLUMN phone VARCHAR(20);
-- Delete a table permanently
DROP TABLE users;
DML — Data Manipulation Language (data)
Insert, update, and delete rows.
-- Insert a row
INSERT INTO users (email, name) VALUES ('alice@example.com', 'Alice');
-- Update rows
UPDATE users SET name = 'Alice Smith' WHERE email = 'alice@example.com';
-- Delete rows
DELETE FROM users WHERE created_at < '2020-01-01';
Warning:
DELETE FROM users;without aWHEREclause deletes all rows. Always double-check.
DQL — Data Query Language (read)
Retrieve data. SELECT is the most-used SQL statement.
-- Get all columns
SELECT * FROM users;
-- Get specific columns
SELECT id, email, name FROM users;
-- Filter rows
SELECT * FROM users WHERE name LIKE 'Alice%';
-- Sort results
SELECT * FROM users ORDER BY created_at DESC;
-- Limit results
SELECT * FROM users LIMIT 10 OFFSET 20;
DCL — Data Control Language (permissions)
Manage who can do what.
-- Grant read access
GRANT SELECT ON users TO analyst_role;
-- Revoke write access
REVOKE INSERT, UPDATE ON orders FROM intern_role;
SELECT deep-dive
SELECT is the command you'll use most. Here's how it maps to everyday questions:
| Question | SQL |
|---|---|
| Show me all users | SELECT * FROM users; |
| How many orders do we have? | SELECT COUNT(*) FROM orders; |
| What's the average order value? | SELECT AVG(total) FROM orders; |
| Which users signed up this month? | SELECT * FROM users WHERE created_at >= DATE_TRUNC('month', NOW()); |
| Top 5 highest-value orders? | SELECT * FROM orders ORDER BY total DESC LIMIT 5; |
| Orders grouped by status | SELECT status, COUNT(*) FROM orders GROUP BY status; |
Aggregate functions
| Function | Purpose | Example |
|---|---|---|
COUNT(*) |
Number of rows | SELECT COUNT(*) FROM users |
SUM(col) |
Total of a column | SELECT SUM(total) FROM orders |
AVG(col) |
Average value | SELECT AVG(price) FROM products |
MIN(col) |
Smallest value | SELECT MIN(price) FROM products |
MAX(col) |
Largest value | SELECT MAX(price) FROM products |
GROUP_CONCAT / STRING_AGG |
Concatenate strings | Varies by database |
JOINs — connecting tables
The real power of SQL comes from combining data from multiple tables.
Consider two tables:
users table orders table
┌────┬──────────────┐ ┌────┬─────────┬───────┐
│ id │ email │ │ id │ user_id │ total │
├────┼──────────────┤ ├────┼─────────┼───────┤
│ 1 │ alice@... │ │ 1 │ 1 │ 50.00 │
│ 2 │ bob@... │ │ 2 │ 1 │ 30.00 │
│ 3 │ carol@... │ │ 3 │ 2 │ 80.00 │
└────┴──────────────┘ └────┴─────────┴───────┘
JOIN types
-- INNER JOIN — only rows that match in both tables
SELECT u.email, o.total
FROM users u
INNER JOIN orders o ON u.id = o.user_id;
-- Returns: alice (×2 orders), bob (×1 order). Carol excluded (no orders).
-- LEFT JOIN — all rows from the left table, nulls if no match
SELECT u.email, o.total
FROM users u
LEFT JOIN orders o ON u.id = o.user_id;
-- Returns: alice (×2), bob (×1), carol (NULL). Shows users with no orders.
-- RIGHT JOIN — all rows from the right table
SELECT u.email, o.total
FROM users u
RIGHT JOIN orders o ON u.id = o.user_id;
-- All orders returned, even if user deleted.
-- FULL OUTER JOIN — all rows from both tables
SELECT u.email, o.total
FROM users u
FULL OUTER JOIN orders o ON u.id = o.user_id;
-- Everything: matched rows + unmatched from both sides.
| JOIN type | Returns | Use when |
|---|---|---|
INNER JOIN |
Only matching rows | You need data that exists in both tables |
LEFT JOIN |
All left + matching right | You want all items even if no related records |
RIGHT JOIN |
Matching left + all right | Less common; swap table order and use LEFT JOIN |
FULL OUTER JOIN |
All rows from both | Finding unmatched rows in either table |
Filtering with WHERE and HAVING
-- WHERE filters rows before aggregation
SELECT * FROM orders WHERE total > 100;
-- HAVING filters groups after GROUP BY
SELECT user_id, SUM(total) as total_spent
FROM orders
GROUP BY user_id
HAVING SUM(total) > 500;
Rule of thumb: use WHERE to filter rows, HAVING to filter aggregated results.
Common WHERE operators
| Operator | Meaning | Example |
|---|---|---|
= |
Equal | WHERE status = 'active' |
!= or <> |
Not equal | WHERE status != 'cancelled' |
>, <, >=, <= |
Comparison | WHERE price > 100 |
BETWEEN |
Range (inclusive) | WHERE price BETWEEN 10 AND 50 |
IN |
Match list | WHERE status IN ('pending', 'processing') |
NOT IN |
Exclude list | WHERE country NOT IN ('US', 'CA') |
LIKE |
Pattern match | WHERE name LIKE 'A%' |
ILIKE |
Case-insensitive LIKE (PostgreSQL) | WHERE name ILIKE 'alice%' |
IS NULL |
Check for null | WHERE deleted_at IS NULL |
IS NOT NULL |
Check non-null | WHERE email IS NOT NULL |
Subqueries
A query inside another query.
-- Find users who have placed at least one order
SELECT * FROM users
WHERE id IN (SELECT DISTINCT user_id FROM orders);
-- Find orders above the average order value
SELECT * FROM orders
WHERE total > (SELECT AVG(total) FROM orders);
-- Correlated subquery: for each user, get their latest order date
SELECT u.email,
(SELECT MAX(created_at) FROM orders WHERE user_id = u.id) AS last_order
FROM users u;
Indexes — making queries fast
Without indexes, the database reads every row to find matches. With the right index, it jumps directly to the data.
-- Create an index on the email column
CREATE INDEX idx_users_email ON users (email);
-- Composite index (covers both columns)
CREATE INDEX idx_orders_user_status ON orders (user_id, status);
-- Unique index (also enforces uniqueness)
CREATE UNIQUE INDEX idx_users_email_unique ON users (email);
| When to index | When NOT to index |
|---|---|
Columns in WHERE, JOIN ON, ORDER BY |
Small tables (full scan is faster) |
| Foreign key columns | Columns rarely used in queries |
| High-cardinality columns (many unique values) | Columns updated very frequently |
| Columns used for sorting/filtering in dashboards | Every column by default |
Rule: indexes speed up reads but slow down writes. Add them where queries are slow, not pre-emptively on every column.
Transactions — keeping data consistent
A transaction groups multiple statements into an all-or-nothing unit.
BEGIN;
UPDATE accounts SET balance = balance - 500 WHERE id = 1; -- debit Alice
UPDATE accounts SET balance = balance + 500 WHERE id = 2; -- credit Bob
COMMIT; -- make both changes permanent
-- or ROLLBACK; -- undo everything if something went wrong
If the server crashes between the two UPDATE statements without transactions, Alice loses £500 but Bob never receives it. Transactions prevent this.
ACID properties:
| Property | Meaning |
|---|---|
| Atomicity | All statements succeed or none do |
| Consistency | Database always moves from one valid state to another |
| Isolation | Concurrent transactions don't interfere with each other |
| Durability | Committed changes survive crashes |
Popular SQL databases
SQL is a standard — many different database systems implement it (with minor variations):
| Database | Best for | Free? | Notes |
|---|---|---|---|
| PostgreSQL | General-purpose, advanced features | Yes (open-source) | Recommended default for new projects |
| MySQL | Web apps, WordPress, high read traffic | Yes (open-source) | Most widely deployed, owned by Oracle |
| MariaDB | MySQL-compatible alternative | Yes (open-source) | Fork of MySQL, often used in Linux distros |
| SQLite | Embedded, mobile apps, local dev | Yes (public domain) | Single file, no server process needed |
| SQL Server | Enterprise Windows environments | Partly (Express free) | Microsoft product, excellent tooling |
| Oracle Database | Large enterprises, financial systems | No | Feature-rich, expensive, complex licensing |
| Amazon Aurora | Cloud-native scalable apps | Pay-per-use | MySQL/PostgreSQL-compatible, managed |
| BigQuery | Data warehousing, analytics | Pay-per-use | Serverless, petabyte-scale |
| Snowflake | Data warehousing, analytics | Pay-per-use | Multi-cloud, separation of compute/storage |
| ClickHouse | Real-time analytics, logs | Yes (open-source) | Columnar, extremely fast aggregations |
SQL vs NoSQL
| SQL (Relational) | NoSQL | |
|---|---|---|
| Structure | Fixed schema, tables with rows/columns | Flexible (document, key-value, graph, wide-column) |
| Query language | SQL (standardised) | Varies per database (MongoDB MQL, Redis commands…) |
| Relationships | JOIN across tables | Usually embed or reference manually |
| ACID transactions | Built-in, battle-tested | Varies — some support it, many don't |
| Scaling | Vertical (bigger machine) + read replicas | Horizontal (more machines) more naturally |
| Best for | Structured business data, reporting, strong consistency | High-volume unstructured data, flexible schemas, caching |
| Examples | PostgreSQL, MySQL, SQLite | MongoDB, Redis, Cassandra, DynamoDB |
Use SQL when: you have structured data, need complex queries and joins, require ACID guarantees, or are building a standard web/mobile app backend.
Use NoSQL when: schema changes rapidly, you need to store unstructured/semi-structured data, you need extreme horizontal scalability, or you need ultra-low latency caching.
SQL in 5 minutes — hands-on example
Let's build a mini e-commerce database from scratch.
-- 1. Create tables
CREATE TABLE products (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
price NUMERIC(10,2) NOT NULL,
stock INT DEFAULT 0
);
CREATE TABLE customers (
id SERIAL PRIMARY KEY,
email TEXT NOT NULL UNIQUE,
name TEXT NOT NULL
);
CREATE TABLE orders (
id SERIAL PRIMARY KEY,
customer_id INT REFERENCES customers(id),
product_id INT REFERENCES products(id),
quantity INT NOT NULL,
created_at TIMESTAMP DEFAULT NOW()
);
-- 2. Insert data
INSERT INTO products (name, price, stock) VALUES
('Laptop', 999.00, 50),
('Mouse', 29.99, 200),
('Keyboard', 79.99, 150);
INSERT INTO customers (email, name) VALUES
('alice@example.com', 'Alice'),
('bob@example.com', 'Bob');
INSERT INTO orders (customer_id, product_id, quantity) VALUES
(1, 1, 1), -- Alice buys 1 Laptop
(1, 2, 2), -- Alice buys 2 Mice
(2, 3, 1); -- Bob buys 1 Keyboard
-- 3. Query: what did each customer spend?
SELECT
c.name,
SUM(p.price * o.quantity) AS total_spent
FROM orders o
JOIN customers c ON o.customer_id = c.id
JOIN products p ON o.product_id = p.id
GROUP BY c.name
ORDER BY total_spent DESC;
-- Result:
-- Alice | 1059.98
-- Bob | 79.99
How to run SQL right now
You don't need to install anything to try SQL:
| Option | How | Best for |
|---|---|---|
| SQLiteOnline.com | Browser, no install | Quick experiments |
| DB Fiddle | Browser, pick PostgreSQL/MySQL/SQLite | Testing queries |
| PostgreSQL playground | Browser | Learning PostgreSQL |
| SQLite + VS Code | Install SQLite extension | Local dev, lightweight |
| PostgreSQL locally | brew install postgresql (macOS) |
Full local setup |
| Docker | docker run -e POSTGRES_PASSWORD=pw postgres |
Isolated, matches production |
SQL in your application
Every major programming language has libraries to run SQL:
# Python with psycopg2 (PostgreSQL)
import psycopg2
conn = psycopg2.connect("dbname=mydb user=alice password=secret")
cur = conn.cursor()
cur.execute("SELECT id, email FROM users WHERE name = %s", ("Alice",))
rows = cur.fetchall()
for row in rows:
print(row)
conn.close()
// Node.js with pg (PostgreSQL)
const { Pool } = require('pg');
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const result = await pool.query(
'SELECT id, email FROM users WHERE name = $1',
['Alice']
);
console.log(result.rows);
# Python with SQLAlchemy ORM (abstracts the SQL)
from sqlalchemy import create_engine, text
engine = create_engine("postgresql://alice:secret@localhost/mydb")
with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM users WHERE name = :name"), {"name": "Alice"})
for row in result:
print(row)
Security note: always use parameterised queries (placeholders like
%sor$1) — never string-concatenate user input into SQL. String concatenation is the #1 cause of SQL injection attacks.
Learning path
| Stage | What to learn | Resources |
|---|---|---|
| Beginner | SELECT, WHERE, ORDER BY, LIMIT, INSERT, UPDATE, DELETE | SQLBolt, W3Schools SQL |
| Intermediate | JOINs, GROUP BY, HAVING, subqueries, indexes | Mode SQL Tutorial, PostgreSQL docs |
| Advanced | Window functions, CTEs, transactions, query optimisation, EXPLAIN | Use The Index Luke, PostgreSQL wiki |
| Practice | Real problems with real datasets | LeetCode SQL, HackerRank SQL, Kaggle datasets |
Common SQL mistakes
| Mistake | Problem | Fix |
|---|---|---|
SELECT * everywhere |
Fetches unnecessary columns, slower, brittle | List only the columns you need |
Missing WHERE in DELETE/UPDATE |
Modifies every row in the table | Always add a WHERE clause; test with SELECT first |
| No indexes on JOIN/WHERE columns | Full table scans, slow queries | Add indexes on foreign keys and frequently-filtered columns |
| String-concatenating user input | SQL injection vulnerability | Use parameterised queries / prepared statements |
| Storing prices as FLOAT | Floating-point rounding errors | Use NUMERIC or DECIMAL for money |
| N+1 queries | 1 query per row = 1,001 DB round trips for 1,000 users | Use JOIN or IN to batch fetches |
| No transactions for multi-step writes | Partial updates on failure | Wrap related writes in BEGIN / COMMIT |
Ignoring NULL behaviour |
NULL != NULL evaluates to NULL, not TRUE |
Use IS NULL / IS NOT NULL; understand NULL semantics |
SQL vs related concepts
| Term | What it is | Relationship to SQL |
|---|---|---|
| SQL | Language | What you write |
| Database | Software (PostgreSQL, MySQL) | What runs SQL |
| Schema | Structure definition | Created using SQL DDL |
| ORM | Code library (Prisma, SQLAlchemy) | Generates SQL automatically |
| NoSQL | Non-relational databases | Alternative to SQL databases |
| NewSQL | Distributed SQL (CockroachDB, Spanner) | SQL language, distributed architecture |
| JDBC / ODBC | Database driver standards | How apps connect to run SQL |
| Query builder | Code library (Knex, Drizzle) | Builds SQL programmatically without full ORM |
Frequently asked questions
Is SQL a programming language?
SQL is a domain-specific language (DSL), not a general-purpose programming language. You can't build a web server in SQL. It's purpose-built for querying and managing relational data. Most developers consider it an essential skill alongside a general-purpose language.
Is SQL hard to learn?
The basics (SELECT, WHERE, JOIN) can be learned in a weekend. You'll be able to query real datasets in a few days. Mastering query optimisation, indexing strategies, and advanced features takes months to years — but you can be productive very quickly.
Which SQL database should I learn first?
PostgreSQL. It's free, open-source, follows the SQL standard closely, runs everywhere (local, Docker, all major cloud providers), and has excellent features. Skills transfer directly to MySQL, SQLite, and others with minor syntax adjustments.
Do I need SQL if I use an ORM?
Yes. ORMs are great for common CRUD operations, but you'll still need SQL to debug slow queries, write complex analytics, perform migrations, or understand what your ORM is generating. Understanding SQL makes you a better developer even when you're not writing it directly.
What's the difference between SQL and MySQL?
SQL is the language. MySQL is a database system that uses SQL. Other databases that use SQL include PostgreSQL, SQLite, and SQL Server. Saying "I know MySQL" means you know a specific database product. Saying "I know SQL" means you know the language.
How is SQL used in data science and analytics?
SQL is the most common tool for data extraction and transformation. Data scientists use it to query databases, join datasets, compute aggregates, and prepare data before analysing it in Python or R. Many analytics roles require SQL proficiency.
Quick start summary
-- 1. Create a table
CREATE TABLE products (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
price NUMERIC(10,2) NOT NULL
);
-- 2. Insert data
INSERT INTO products (name, price) VALUES ('Widget', 9.99), ('Gadget', 49.99);
-- 3. Read data
SELECT * FROM products WHERE price < 20 ORDER BY price;
-- 4. Update a row
UPDATE products SET price = 8.99 WHERE name = 'Widget';
-- 5. Delete a row
DELETE FROM products WHERE price > 40;
-- 6. Join tables (once you have a second table)
SELECT p.name, c.category
FROM products p
JOIN categories c ON p.category_id = c.id;
SQL has been powering applications for 50+ years — and there's no sign of it slowing down. Whether you're building web apps, doing data analysis, or engineering data pipelines, SQL is one of the highest-leverage skills you can learn.