1. Introduction to Greedy Algorithms
1.1 General Greedy Method
function GREEDY(problem):
solution = empty
while problem not solved:
// SELECT: Choose the best available option
choice = SELECT_BEST_OPTION(problem)
// FEASIBILITY: Check if choice is valid
if IS_FEASIBLE(solution, choice):
// INCLUDE: Add choice to solution
solution = solution + choice
return solution
1.2 Components of Greedy Algorithm
- Candidate Set: Set of all possible choices
- Selection Function: Chooses the best candidate
- Feasibility Function: Checks if candidate can be included
- Objective Function: Assigns value to solution
- Solution Function: Checks if complete solution is found
2. Properties of Greedy Algorithms
2.1 When Does Greedy Work?
A problem can be solved optimally using greedy approach if it has two properties:
2.2 Greedy vs Other Approaches
| Aspect | Greedy | Dynamic Programming | Backtracking |
|---|---|---|---|
| Approach | Make best local choice | Try all subproblems | Try all paths, backtrack |
| Reconsideration | Never | Yes (stores all) | Yes (backtracks) |
| Optimality | Sometimes | Always (if applicable) | Always (exhaustive) |
| Efficiency | Fast | Moderate | Can be slow |
- Fractional Knapsack (works)
- Activity Selection (works)
- Huffman Coding (works)
- Minimum Spanning Tree (works)
- Single Source Shortest Path - Dijkstra (works)
- 0/1 Knapsack (does NOT work - needs DP)
3. Fractional Knapsack Problem
3.1 Greedy Strategy
Key Insight: Calculate profit-to-weight ratio (profit/weight) for each item. Select items in decreasing order of this ratio.
- Calculate ratio = profit[i] / weight[i] for each item
- Sort items by ratio in decreasing order
- For each item (in sorted order):
- If item fits completely, take it entirely
- Else, take the fraction that fits
- Continue until knapsack is full
function FRACTIONAL_KNAPSACK(W, weights[], profits[], n):
// Calculate ratios and sort
for i = 1 to n:
ratio[i] = profits[i] / weights[i]
// Sort items by ratio in decreasing order
SORT_BY_RATIO_DESCENDING(items)
totalProfit = 0
remainingCapacity = W
for each item in sorted order:
if weights[item] <= remainingCapacity:
// Take complete item
totalProfit += profits[item]
remainingCapacity -= weights[item]
else:
// Take fraction of item
fraction = remainingCapacity / weights[item]
totalProfit += profits[item] * fraction
remainingCapacity = 0
break
return totalProfit
n = 7, m = 15
(p1...p7) = (10, 5, 15, 7, 6, 18, 3)
(w1...w7) = (2, 3, 5, 7, 1, 4, 1)
Step 1: Calculate Profit/Weight Ratio
| Item | Profit | Weight | Ratio (p/w) |
|---|---|---|---|
| 1 | 10 | 2 | 5.0 |
| 2 | 5 | 3 | 1.67 |
| 3 | 15 | 5 | 3.0 |
| 4 | 7 | 7 | 1.0 |
| 5 | 6 | 1 | 6.0 |
| 6 | 18 | 4 | 4.5 |
| 7 | 3 | 1 | 3.0 |
Step 2: Sort by Ratio (Descending)
Order: Item 5 (6.0) โ Item 1 (5.0) โ Item 6 (4.5) โ Item 3 (3.0) โ Item 7 (3.0) โ Item 2 (1.67) โ Item 4 (1.0)
Step 3: Fill Knapsack (Capacity = 15)
| Item | Weight | Fraction | Profit Added | Remaining Capacity |
|---|---|---|---|---|
| 5 | 1 | 1 (full) | 6 | 15 - 1 = 14 |
| 1 | 2 | 1 (full) | 10 | 14 - 2 = 12 |
| 6 | 4 | 1 (full) | 18 | 12 - 4 = 8 |
| 3 | 5 | 1 (full) | 15 | 8 - 5 = 3 |
| 7 | 1 | 1 (full) | 3 | 3 - 1 = 2 |
| 2 | 3 | 2/3 (fraction) | 5 ร 2/3 = 3.33 | 2 - 2 = 0 |
3.2 Time Complexity Analysis
| Operation | Time |
|---|---|
| Calculate ratios | O(n) |
| Sort by ratio | O(n log n) |
| Fill knapsack | O(n) |
| Total | O(n log n) |
4. Prim's Algorithm (MST)
4.1 Algorithm
- Initialize MST with any starting vertex
- Mark starting vertex as visited
- While MST has fewer than V-1 edges:
- Find minimum weight edge (u,v) where u is in MST and v is not
- Add edge (u,v) to MST
- Mark v as visited
- Return MST
function PRIMS(graph, start):
MST = empty set
visited = {start}
edges = all edges from start
while |visited| < |V|:
// Find minimum weight edge to unvisited vertex
(u, v, weight) = MIN_EDGE where u in visited, v not in visited
MST.add((u, v, weight))
visited.add(v)
// Add all edges from v to unvisited vertices
for each edge (v, w) where w not in visited:
edges.add((v, w))
return MST
A ---4--- B
/| /|
2 | 1 |
/ | / |
C 3 D 5
\ | / |
6 | 3 |
\| / |
E ---2--- F
Step-by-Step Execution (Start from A):
| Step | Visited | Available Edges | Selected Edge | MST Weight |
|---|---|---|---|---|
| 1 | {A} | A-B(4), A-C(2), A-E(3) | A-C (2) | 2 |
| 2 | {A,C} | A-B(4), A-E(3), C-E(6) | A-E (3) | 5 |
| 3 | {A,C,E} | A-B(4), E-D(3), E-F(2) | E-F (2) | 7 |
| 4 | {A,C,E,F} | A-B(4), E-D(3), F-B(5) | E-D (3) | 10 |
| 5 | {A,C,E,F,D} | A-B(4), F-B(5), D-B(1) | D-B (1) | 11 |
A B
/ \
2 1
/ \
C D-----
\ /
\ 3
\ /
E ---2--- F
MST Edges: A-C(2), A-E(3), E-F(2), E-D(3), D-B(1)
4.2 Time Complexity
| Implementation | Time Complexity |
|---|---|
| Adjacency Matrix | O(Vยฒ) |
| Binary Heap + Adjacency List | O(E log V) |
| Fibonacci Heap | O(E + V log V) |
5. Kruskal's Algorithm (MST)
5.1 Algorithm
- Sort all edges by weight in ascending order
- Initialize MST as empty
- For each edge (u, v) in sorted order:
- If adding (u, v) doesn't create a cycle:
- Add (u, v) to MST
- Stop when MST has V-1 edges
function KRUSKALS(graph):
MST = empty set
edges = all edges sorted by weight ascending
// Initialize Union-Find (each vertex in own set)
for each vertex v:
MAKE_SET(v)
for each edge (u, v, weight) in sorted edges:
if FIND(u) โ FIND(v): // Different sets = no cycle
MST.add((u, v, weight))
UNION(u, v)
if |MST| == |V| - 1:
break
return MST
Step 1: Sort Edges by Weight
D-B(1), A-C(2), E-F(2), A-E(3), E-D(3), A-B(4), F-B(5), C-E(6)
Step 2: Process Edges
| Edge | Weight | Action | Reason |
|---|---|---|---|
| D-B | 1 | ADD | No cycle |
| A-C | 2 | ADD | No cycle |
| E-F | 2 | ADD | No cycle |
| A-E | 3 | ADD | Connects {A,C} to {E,F} |
| E-D | 3 | ADD | Connects to {D,B} |
| A-B | 4 | SKIP | Would create cycle |
MST has 5 edges (V-1), STOP.
5.2 Prim's vs Kruskal's
Prim's Algorithm
- Grows MST from a vertex
- Vertex-based approach
- Better for dense graphs
- O(E log V) with binary heap
- Always maintains connected tree
Kruskal's Algorithm
- Picks edges globally
- Edge-based approach
- Better for sparse graphs
- O(E log E) = O(E log V)
- May have forest initially
5.3 Time Complexity
| Operation | Time |
|---|---|
| Sort edges | O(E log E) |
| Union-Find operations | O(E ฮฑ(V)) โ O(E) |
| Total | O(E log E) = O(E log V) |
6. Job Sequencing with Deadlines
6.1 Greedy Strategy
Key Insight: Sort jobs by profit in decreasing order. For each job, schedule it in the latest available slot before its deadline.
- Sort jobs by profit in decreasing order
- Find maximum deadline (determines time slots)
- Initialize all slots as empty
- For each job (in sorted order):
- Find the latest empty slot โค job's deadline
- If slot found, schedule job in that slot
- If no slot, skip the job
- Calculate total profit of scheduled jobs
n = 7, Profits (p1...p7) = (3, 5, 20, 18, 1, 6, 30)
Deadlines (d1...d7) = (1, 3, 4, 3, 2, 1, 2)
Step 1: Sort by Profit (Descending)
| Job | Profit | Deadline |
|---|---|---|
| J7 | 30 | 2 |
| J3 | 20 | 4 |
| J4 | 18 | 3 |
| J6 | 6 | 1 |
| J2 | 5 | 3 |
| J1 | 3 | 1 |
| J5 | 1 | 2 |
Step 2: Maximum Deadline = 4, so 4 time slots
Step 3: Schedule Jobs
| Job | Deadline | Try Slots | Assigned | Timeline |
|---|---|---|---|---|
| J7 (30) | 2 | 2,1 | Slot 2 | [_,J7,_,_] |
| J3 (20) | 4 | 4,3,2,1 | Slot 4 | [_,J7,_,J3] |
| J4 (18) | 3 | 3,2,1 | Slot 3 | [_,J7,J4,J3] |
| J6 (6) | 1 | 1 | Slot 1 | [J6,J7,J4,J3] |
| J2 (5) | 3 | 3,2,1 | None (all full) | [J6,J7,J4,J3] |
| J1 (3) | 1 | 1 | None (full) | [J6,J7,J4,J3] |
| J5 (1) | 2 | 2,1 | None (full) | [J6,J7,J4,J3] |
Final Schedule:
J6 (6)
J7 (30)
J4 (18)
J3 (20)
6.2 Time Complexity
| Operation | Time |
|---|---|
| Sort jobs | O(n log n) |
| Schedule each job | O(n) per job |
| Total (basic) | O(nยฒ) |
| With Union-Find | O(n log n) |
7. Optimal Storage on Tapes
7.1 Understanding the Problem
Time to access program i = lโ + lโ + ... + lแตข
7.2 Greedy Strategy
Key Insight: Store programs in increasing order of length. Shorter programs first means they contribute less to the cumulative access time of programs stored after them.
- Sort programs by length in ascending order
- Store programs in sorted order on tape
- This minimizes mean retrieval time
Order 1: 5, 10, 3
- Access 5: 5
- Access 10: 5 + 10 = 15
- Access 3: 5 + 10 + 3 = 18
- Total = 5 + 15 + 18 = 38, MRT = 38/3 = 12.67
Order 2: 3, 5, 10 (Optimal - sorted by length)
- Access 3: 3
- Access 5: 3 + 5 = 8
- Access 10: 3 + 5 + 10 = 18
- Total = 3 + 8 + 18 = 29, MRT = 29/3 = 9.67
7.3 With Frequencies
If programs have different access frequencies, sort by length/frequency ratio in ascending order (or frequency/length in descending order).
7.4 Time Complexity
O(n log n) - dominated by sorting
8. Practice Questions
n=5, M=100, (p1..p5)=(10,20,30,40,50), (w1..w5)=(20,30,66,40,60)
Ratios: 0.5, 0.67, 0.45, 1.0, 0.83
Sorted order: Item 4, Item 5, Item 2, Item 1, Item 3
Selection:
- Item 4: weight 40, profit 40, remaining = 60
- Item 5: weight 60, profit 50, remaining = 0
Maximum Profit = 90
A greedy algorithm makes locally optimal choices at each step, hoping to find global optimum.
General structure:
- Initialize solution as empty
- While solution not complete:
- Select best available choice
- If feasible, add to solution
- Return solution
Prim's: Vertex-based, grows from single vertex, O(E log V), better for dense graphs.
Kruskal's: Edge-based, global sorting, O(E log E), better for sparse graphs.
Preference: Prim's when E โ Vยฒ, Kruskal's when E โ V.
Sorted by profit: J1(100), J4(27), J3(15), J2(10)
Max deadline = 2, slots = [_, _]
- J1 (d=2): Slot 2 โ [_, J1]
- J4 (d=1): Slot 1 โ [J4, J1]
- J3, J2: No slots available
Maximum Profit = 100 + 27 = 127
Key Points Summary
- Greedy: Make locally optimal choice, never backtrack
- Requires: Greedy Choice Property + Optimal Substructure
- Fractional Knapsack: Sort by profit/weight ratio, O(n log n)
- 0/1 Knapsack: Greedy FAILS - use DP instead
- Prim's: Vertex-based MST, grows from one vertex
- Kruskal's: Edge-based MST, sorts all edges, uses Union-Find
- Both MST algorithms: O(E log V)
- Job Sequencing: Sort by profit, fill latest available slot
- Optimal Storage: Sort by length ascending (shortest first)