Module III

Greedy Algorithms & Applications

Master the greedy approach with Fractional Knapsack, Minimum Spanning Trees (Prim's and Kruskal's), Job Sequencing, and Optimal Storage on Tapes.

1. Introduction to Greedy Algorithms

Greedy Algorithm
A Greedy Algorithm makes the locally optimal choice at each step, hoping that these local choices lead to a globally optimal solution. It never reconsiders its choices (no backtracking).
๐ŸŽฏ
Exam Tip (100% Frequency): The Knapsack problem appears in EVERY paper. You must know both Greedy (Fractional) and DP (0/1) versions, and understand WHY greedy works for fractional but fails for 0/1.

1.1 General Greedy Method

Pseudocode
function GREEDY(problem):
    solution = empty
    while problem not solved:
        // SELECT: Choose the best available option
        choice = SELECT_BEST_OPTION(problem)

        // FEASIBILITY: Check if choice is valid
        if IS_FEASIBLE(solution, choice):
            // INCLUDE: Add choice to solution
            solution = solution + choice

    return solution

1.2 Components of Greedy Algorithm

  1. Candidate Set: Set of all possible choices
  2. Selection Function: Chooses the best candidate
  3. Feasibility Function: Checks if candidate can be included
  4. Objective Function: Assigns value to solution
  5. Solution Function: Checks if complete solution is found

2. Properties of Greedy Algorithms

2.1 When Does Greedy Work?

A problem can be solved optimally using greedy approach if it has two properties:

1
Greedy Choice Property
A globally optimal solution can be arrived at by making locally optimal (greedy) choices. At each step, we can make a choice that looks best at the moment without considering future consequences.
2
Optimal Substructure
An optimal solution to the problem contains optimal solutions to its subproblems. After making a greedy choice, we're left with a smaller subproblem of the same type.

2.2 Greedy vs Other Approaches

Aspect Greedy Dynamic Programming Backtracking
Approach Make best local choice Try all subproblems Try all paths, backtrack
Reconsideration Never Yes (stores all) Yes (backtracks)
Optimality Sometimes Always (if applicable) Always (exhaustive)
Efficiency Fast Moderate Can be slow
๐Ÿ“Œ Common Greedy Problems
  • Fractional Knapsack (works)
  • Activity Selection (works)
  • Huffman Coding (works)
  • Minimum Spanning Tree (works)
  • Single Source Shortest Path - Dijkstra (works)
  • 0/1 Knapsack (does NOT work - needs DP)

3. Fractional Knapsack Problem

Fractional Knapsack Problem
Given n items with weights and profits, and a knapsack of capacity W, select items to maximize total profit. Items can be broken into fractions.

3.1 Greedy Strategy

Key Insight: Calculate profit-to-weight ratio (profit/weight) for each item. Select items in decreasing order of this ratio.

๐Ÿ“‹ Fractional Knapsack Algorithm
  1. Calculate ratio = profit[i] / weight[i] for each item
  2. Sort items by ratio in decreasing order
  3. For each item (in sorted order):
    • If item fits completely, take it entirely
    • Else, take the fraction that fits
  4. Continue until knapsack is full
Pseudocode
function FRACTIONAL_KNAPSACK(W, weights[], profits[], n):
    // Calculate ratios and sort
    for i = 1 to n:
        ratio[i] = profits[i] / weights[i]

    // Sort items by ratio in decreasing order
    SORT_BY_RATIO_DESCENDING(items)

    totalProfit = 0
    remainingCapacity = W

    for each item in sorted order:
        if weights[item] <= remainingCapacity:
            // Take complete item
            totalProfit += profits[item]
            remainingCapacity -= weights[item]
        else:
            // Take fraction of item
            fraction = remainingCapacity / weights[item]
            totalProfit += profits[item] * fraction
            remainingCapacity = 0
            break

    return totalProfit
Solved Example (Exam Pattern - 100% Frequency) 10 Marks
Obtain the solution to the following knapsack problem using Greedy method:
n = 7, m = 15
(p1...p7) = (10, 5, 15, 7, 6, 18, 3)
(w1...w7) = (2, 3, 5, 7, 1, 4, 1)
โœ“ Solution

Step 1: Calculate Profit/Weight Ratio

Item Profit Weight Ratio (p/w)
1 10 2 5.0
2 5 3 1.67
3 15 5 3.0
4 7 7 1.0
5 6 1 6.0
6 18 4 4.5
7 3 1 3.0

Step 2: Sort by Ratio (Descending)

Order: Item 5 (6.0) โ†’ Item 1 (5.0) โ†’ Item 6 (4.5) โ†’ Item 3 (3.0) โ†’ Item 7 (3.0) โ†’ Item 2 (1.67) โ†’ Item 4 (1.0)

Step 3: Fill Knapsack (Capacity = 15)

Item Weight Fraction Profit Added Remaining Capacity
5 1 1 (full) 6 15 - 1 = 14
1 2 1 (full) 10 14 - 2 = 12
6 4 1 (full) 18 12 - 4 = 8
3 5 1 (full) 15 8 - 5 = 3
7 1 1 (full) 3 3 - 1 = 2
2 3 2/3 (fraction) 5 ร— 2/3 = 3.33 2 - 2 = 0
Maximum Profit = 6 + 10 + 18 + 15 + 3 + 3.33 = 55.33
Solution: x = (1, 2/3, 1, 0, 1, 1, 1)

3.2 Time Complexity Analysis

Operation Time
Calculate ratios O(n)
Sort by ratio O(n log n)
Fill knapsack O(n)
Total O(n log n)
โš ๏ธ
Why Greedy Fails for 0/1 Knapsack: In 0/1 Knapsack, items cannot be broken. Greedy may leave space unfilled, missing better combinations. Example: W=50, items (60,10), (100,20), (120,30). Greedy takes item 1 first (ratio 6), leaving space. Optimal is items 2+3.

4. Prim's Algorithm (MST)

Minimum Spanning Tree (MST)
A spanning tree of a connected graph G is a subgraph that is a tree and connects all vertices. The MST has the minimum total edge weight among all spanning trees.
Prim's Algorithm
Prim's algorithm grows the MST from a starting vertex by repeatedly adding the minimum weight edge that connects a vertex in the tree to a vertex outside the tree.

4.1 Algorithm

๐Ÿ“‹ Prim's Algorithm Steps
  1. Initialize MST with any starting vertex
  2. Mark starting vertex as visited
  3. While MST has fewer than V-1 edges:
    • Find minimum weight edge (u,v) where u is in MST and v is not
    • Add edge (u,v) to MST
    • Mark v as visited
  4. Return MST
Pseudocode
function PRIMS(graph, start):
    MST = empty set
    visited = {start}
    edges = all edges from start

    while |visited| < |V|:
        // Find minimum weight edge to unvisited vertex
        (u, v, weight) = MIN_EDGE where u in visited, v not in visited

        MST.add((u, v, weight))
        visited.add(v)

        // Add all edges from v to unvisited vertices
        for each edge (v, w) where w not in visited:
            edges.add((v, w))

    return MST
Solved Example (Exam Pattern - 67% Frequency) 10 Marks
Find MST using Prim's Algorithm for the following graph starting from vertex A:
Graph
        A ---4--- B
       /|        /|
      2 |       1 |
     /  |      /  |
    C   3     D   5
     \  |    /    |
      6 |   3     |
       \| /       |
        E ---2--- F
โœ“ Solution

Step-by-Step Execution (Start from A):

Step Visited Available Edges Selected Edge MST Weight
1 {A} A-B(4), A-C(2), A-E(3) A-C (2) 2
2 {A,C} A-B(4), A-E(3), C-E(6) A-E (3) 5
3 {A,C,E} A-B(4), E-D(3), E-F(2) E-F (2) 7
4 {A,C,E,F} A-B(4), E-D(3), F-B(5) E-D (3) 10
5 {A,C,E,F,D} A-B(4), F-B(5), D-B(1) D-B (1) 11
Minimum Spanning Tree
        A        B
       /          \
      2            1
     /              \
    C         D-----
     \       /
      \     3
       \   /
        E ---2--- F

MST Edges: A-C(2), A-E(3), E-F(2), E-D(3), D-B(1)
Total MST Weight = 2 + 3 + 2 + 3 + 1 = 11

4.2 Time Complexity

Implementation Time Complexity
Adjacency Matrix O(Vยฒ)
Binary Heap + Adjacency List O(E log V)
Fibonacci Heap O(E + V log V)

5. Kruskal's Algorithm (MST)

Kruskal's Algorithm
Kruskal's algorithm builds the MST by sorting all edges by weight and adding them one by one, skipping edges that would create a cycle. Uses Union-Find (Disjoint Set) data structure for cycle detection.

5.1 Algorithm

๐Ÿ“‹ Kruskal's Algorithm Steps
  1. Sort all edges by weight in ascending order
  2. Initialize MST as empty
  3. For each edge (u, v) in sorted order:
    • If adding (u, v) doesn't create a cycle:
    • Add (u, v) to MST
  4. Stop when MST has V-1 edges
Pseudocode
function KRUSKALS(graph):
    MST = empty set
    edges = all edges sorted by weight ascending

    // Initialize Union-Find (each vertex in own set)
    for each vertex v:
        MAKE_SET(v)

    for each edge (u, v, weight) in sorted edges:
        if FIND(u) โ‰  FIND(v):  // Different sets = no cycle
            MST.add((u, v, weight))
            UNION(u, v)

        if |MST| == |V| - 1:
            break

    return MST
Solved Example (Exam Pattern - 67% Frequency) 10 Marks
Find MST using Kruskal's Algorithm for the same graph:
โœ“ Solution

Step 1: Sort Edges by Weight

D-B(1), A-C(2), E-F(2), A-E(3), E-D(3), A-B(4), F-B(5), C-E(6)

Step 2: Process Edges

Edge Weight Action Reason
D-B 1 ADD No cycle
A-C 2 ADD No cycle
E-F 2 ADD No cycle
A-E 3 ADD Connects {A,C} to {E,F}
E-D 3 ADD Connects to {D,B}
A-B 4 SKIP Would create cycle

MST has 5 edges (V-1), STOP.

MST Edges: D-B(1), A-C(2), E-F(2), A-E(3), E-D(3)
Total Weight = 1 + 2 + 2 + 3 + 3 = 11

5.2 Prim's vs Kruskal's

Prim's Algorithm

  • Grows MST from a vertex
  • Vertex-based approach
  • Better for dense graphs
  • O(E log V) with binary heap
  • Always maintains connected tree

Kruskal's Algorithm

  • Picks edges globally
  • Edge-based approach
  • Better for sparse graphs
  • O(E log E) = O(E log V)
  • May have forest initially

5.3 Time Complexity

Operation Time
Sort edges O(E log E)
Union-Find operations O(E ฮฑ(V)) โ‰ˆ O(E)
Total O(E log E) = O(E log V)

6. Job Sequencing with Deadlines

Job Sequencing Problem
Given n jobs with deadlines and profits, where each job takes 1 unit of time, schedule jobs to maximize total profit. A job must be completed before or on its deadline.

6.1 Greedy Strategy

Key Insight: Sort jobs by profit in decreasing order. For each job, schedule it in the latest available slot before its deadline.

๐Ÿ“‹ Job Sequencing Algorithm
  1. Sort jobs by profit in decreasing order
  2. Find maximum deadline (determines time slots)
  3. Initialize all slots as empty
  4. For each job (in sorted order):
    • Find the latest empty slot โ‰ค job's deadline
    • If slot found, schedule job in that slot
    • If no slot, skip the job
  5. Calculate total profit of scheduled jobs
Solved Example (Exam Pattern - 50% Frequency) 10 Marks
Solve the following Job Sequencing problem:
n = 7, Profits (p1...p7) = (3, 5, 20, 18, 1, 6, 30)
Deadlines (d1...d7) = (1, 3, 4, 3, 2, 1, 2)
โœ“ Solution

Step 1: Sort by Profit (Descending)

Job Profit Deadline
J7 30 2
J3 20 4
J4 18 3
J6 6 1
J2 5 3
J1 3 1
J5 1 2

Step 2: Maximum Deadline = 4, so 4 time slots

Step 3: Schedule Jobs

Job Deadline Try Slots Assigned Timeline
J7 (30) 2 2,1 Slot 2 [_,J7,_,_]
J3 (20) 4 4,3,2,1 Slot 4 [_,J7,_,J3]
J4 (18) 3 3,2,1 Slot 3 [_,J7,J4,J3]
J6 (6) 1 1 Slot 1 [J6,J7,J4,J3]
J2 (5) 3 3,2,1 None (all full) [J6,J7,J4,J3]
J1 (3) 1 1 None (full) [J6,J7,J4,J3]
J5 (1) 2 2,1 None (full) [J6,J7,J4,J3]

Final Schedule:

Slot 1
J6 (6)
Slot 2
J7 (30)
Slot 3
J4 (18)
Slot 4
J3 (20)
Maximum Profit = 6 + 30 + 18 + 20 = 74
Jobs scheduled: J6, J7, J4, J3

6.2 Time Complexity

Operation Time
Sort jobs O(n log n)
Schedule each job O(n) per job
Total (basic) O(nยฒ)
With Union-Find O(n log n)

7. Optimal Storage on Tapes

Optimal Storage Problem
Given n programs with lengths l1, l2, ..., ln to be stored on a tape, arrange them to minimize Mean Retrieval Time (MRT). Programs are stored sequentially; to access program i, all programs before it must be read.

7.1 Understanding the Problem

Total Retrieval Time = ฮฃ (time to access program i)
Time to access program i = lโ‚ + lโ‚‚ + ... + lแตข
MRT = Total Retrieval Time / n

7.2 Greedy Strategy

Key Insight: Store programs in increasing order of length. Shorter programs first means they contribute less to the cumulative access time of programs stored after them.

๐Ÿ“‹ Optimal Storage Algorithm
  1. Sort programs by length in ascending order
  2. Store programs in sorted order on tape
  3. This minimizes mean retrieval time
๐Ÿ“ Example: Programs with lengths 5, 10, 3

Order 1: 5, 10, 3

  • Access 5: 5
  • Access 10: 5 + 10 = 15
  • Access 3: 5 + 10 + 3 = 18
  • Total = 5 + 15 + 18 = 38, MRT = 38/3 = 12.67

Order 2: 3, 5, 10 (Optimal - sorted by length)

  • Access 3: 3
  • Access 5: 3 + 5 = 8
  • Access 10: 3 + 5 + 10 = 18
  • Total = 3 + 8 + 18 = 29, MRT = 29/3 = 9.67
Optimal MRT = 9.67 (sorted order)

7.3 With Frequencies

If programs have different access frequencies, sort by length/frequency ratio in ascending order (or frequency/length in descending order).

Sort by lแตข/fแตข (ascending) where fแตข = frequency of access

7.4 Time Complexity

O(n log n) - dominated by sorting

8. Practice Questions

Question 1 (100% Frequency) 10 Marks
Write algorithm for greedy knapsack and solve:
n=5, M=100, (p1..p5)=(10,20,30,40,50), (w1..w5)=(20,30,66,40,60)
View Answer โ–ผ

Ratios: 0.5, 0.67, 0.45, 1.0, 0.83

Sorted order: Item 4, Item 5, Item 2, Item 1, Item 3

Selection:

  • Item 4: weight 40, profit 40, remaining = 60
  • Item 5: weight 60, profit 50, remaining = 0

Maximum Profit = 90

Question 2 (50% Frequency) 5 Marks
What is a greedy algorithm? Write the general abstract algorithm for greedy design method.
View Answer โ–ผ

A greedy algorithm makes locally optimal choices at each step, hoping to find global optimum.

General structure:

  1. Initialize solution as empty
  2. While solution not complete:
  3.   Select best available choice
  4.   If feasible, add to solution
  5. Return solution
Question 3 (67% Frequency) 10 Marks
Compare Prim's and Kruskal's algorithms for finding MST. When would you prefer one over the other?
View Answer โ–ผ

Prim's: Vertex-based, grows from single vertex, O(E log V), better for dense graphs.

Kruskal's: Edge-based, global sorting, O(E log E), better for sparse graphs.

Preference: Prim's when E โ‰ˆ Vยฒ, Kruskal's when E โ‰ˆ V.

Question 4 5 Marks
What is job sequencing with deadline? Solve: n=4, (P1..P4)=(100,10,15,27), (d1..d4)=(2,1,2,1)
View Answer โ–ผ

Sorted by profit: J1(100), J4(27), J3(15), J2(10)

Max deadline = 2, slots = [_, _]

  • J1 (d=2): Slot 2 โ†’ [_, J1]
  • J4 (d=1): Slot 1 โ†’ [J4, J1]
  • J3, J2: No slots available

Maximum Profit = 100 + 27 = 127

Key Points Summary

๐Ÿ“Œ Remember for Exam
  • Greedy: Make locally optimal choice, never backtrack
  • Requires: Greedy Choice Property + Optimal Substructure
  • Fractional Knapsack: Sort by profit/weight ratio, O(n log n)
  • 0/1 Knapsack: Greedy FAILS - use DP instead
  • Prim's: Vertex-based MST, grows from one vertex
  • Kruskal's: Edge-based MST, sorts all edges, uses Union-Find
  • Both MST algorithms: O(E log V)
  • Job Sequencing: Sort by profit, fill latest available slot
  • Optimal Storage: Sort by length ascending (shortest first)