The idea behind recursion (and a hint for when to use recursion) is that some problem P is defined/solved in terms of one or more problems P', which are identical in nature to P but smaller in size. To use recursion, the problem must have:
The examples in this section are designed to help you think about how to apply recursion to a problem. These simple examples are not good examples of when to use recursion, just how to use recursion to solve a problem.
The factorial of a positive integer N is defined as the product of all of the positive integers between 1 and N (inclusive). In other words:
Let's look at our 3 requirements:
A number (X) raised to a positive integer (N) is defined as multiplying X by itself (N-1) times. For example, X2 = X * X.
What is a recursive definition of XN? What would be our 3 requirements:
Many recursive methods are very similar to the underlying definitions. Let's look at Factorial:
Note that negative N generates an exception. Notice also that the method is calling itself and uses the result in the return expression. Why do you think the return type is long instead of int?
What do you think the integer powers recursive method would look like?
Often, students struggle to understand how recursion works. They often wonder how a method is able to call itself and maintain two sets of values for the same local variable. There are two important ideas that allow recursion to work: Activation Record and Run-Time Stack.
An Activation Record is a block of memory allocated to store arguments for method parameters, local variables, and the return address during a method call. An activation record is assocated with each method call, so if a method is called multiple times, multiple records are created. For example, with the factorial method above and this method call:
The activation record might look like this:
The return address is the set by the operating system. It refers to the next instruction to execute once the method call is finished. In this particular example, the next instruction would probably be something like "save the returned value into the variable result".
The Run-Time Stack is an area of the computer's memory which maintains activation records in Last In First Out (LIFO) order. Recall our previous discussion of Stacks.
When a method is called, an activation record containing the parameters, return address, and local variables is pushed onto the top of the run-time stack. If the method subsequently calls itself, a new, distinct activation record containing new data is pushed onto the top of the run-time stack. The activation record at the top of the run-time stack represents the currently-executing call. Activation records below the top represent previous calls that are waiting to be returned to. When the top call terminates, control returns to the address from the top activation record and then the top activation record is popped from the stack. For example, consider this simple class:
Here is an illustration of how the recursion, activation records, and run-time stack works:
| Instruction(s) Executed | Computer Memory After Instructions Executed |
|---|---|
|
Program starts In main: long result; |
|
|
In main: |
|
|
In factorial(3): throw new IllegalArgumentException(); if (N <= 1) return 1; result = factorial(N-1); |
|
|
In factorial(2): throw new IllegalArgumentException(); if (N <= 1) return 1; result = factorial(N-1); |
|
|
In factorial(1): throw new IllegalArgumentException(); if (N <= 1) |
|
|
In factorial(1): return 1; |
|
|
In factorial(2): |
|
|
In factorial(3): |
|
A couple of notes about this illustration:
main method, where the result of the factorial method call is stored into resultfactorial method, where the result of the factorial method call is stored into resultFor another example of recursion in action, see RecursionTrace.java.
Let's look at one more simple example: Sequential Search, where we find a key in an array or linked list by checking each item in sequence. We know how to do this iteratively (see the bag implementations of contains), it's basically a loop to go through each item. Let's now look at how to do it recursively. Remember, we always need to consider the problem in terms of a smaller problem of the same type. What are:
For an implementation of recursive sequential search, see SeqSArray.java.
Let's also consider recursive sequential search of a linked list. What is similar and what is different from the array-based version?
list.getNextNode() == nullFor an implementation of recursive sequential search with a linked list, see SeqSLinked.java (and Node.java).
So far, the recursive algorithms that we have seen (see text for more) are simple, and probably would NOT be done recursively. The iterative solutions work fine and are probably more intuitive and easier to implement. They were just used to demonstrate how recursion works. However, recursion often suggests approaches to problem solving that are more logical and easier than without it. For example, divide and conquer.
The idea behind divide and conquer algorithms is that a problem can be solved by breaking it down to one or more "smaller" problems in a systematic way. The subproblem(s) are usually a fraction of the size of the original problem and are usually identical in nature to the original problem. By doing this, the overall solution is just the result of the smaller subproblems being combined. If this sounds similar to recursion, that's because it's often implemented recursively. The key difference is that divide and conquer makes more than one recursive call at each level (unlike just one in the examples above).
We can think of each lower level as solving the same problem as the level above. The only difference in each level is the size of the problem, which is 1/2 (sometimes some other fraction) of that of the level above it. Note how quickly the problem size is reduced.
Let's look at one of our earlier recursive problems: Power function (XN). We have already seen a simple iterative solution using a for loop. We have also already seen and discussed a simple recursive solution. Note that the recursive solution does recursive calls rather than loop iterations. However both algorithms have the same runtime: we must do O(N) multiplications to complete the problem. Can we come up with a solution that is better in terms of runtime? Let's try Divide and Conquer. We typically need to consider two important things:
For XN, the problem "size" is the exponent N. So, a subproblem would be the same problem with a smaller N. Let's try cutting N in half so each subproblem is of size N/2. This means defining XN in terms of XN/2 (don't forget about the base case(s)). So, how is the original problem solved in terms of XN/2? Let's look at some examples.
The divide and conquer approach could look something like this, where solving XN means computing XN/2 twice (using two recursive calls), then multiplying the results together once the recursive calls return.
|
Possible Implementation public long power(long X, long N) { if (N == 0) //base case return 1; else if (N == 1) //base case return X; else //recursive case { return power(X, N/2) * power(X, N/2); } } |
Trace of Recursive Calls
|
If you look at that tree, you might notice that you're computing XN/2 twice, XN/4 four times, XN/8 eight times, etc. You might think this is wasteful (it is). We're computing things that we already computed! Instead, you might think we could just memorize the first result (e.g. the first XN/2) and use its result any time we need it. This is called memoization, i.e. we are maintaining a memo of previously-calculated results. A memoized version of the approach could look like this:
|
Possible Implementation public long power(long X, long N) { if (N == 0) //base case return 1; else if (N == 1) //base case return X; else //recursive case { long result = power(X, N/2); //memoize return result*result; } } |
Trace of Recursive Calls
|
Is this an improvement over the other approaches seen earlier? The problem size is being cut in half each time, which suggests that it takes on the order of log2(N), or O(log(N)), multiplications. See text 7.25-7.27 for more thorough analysis, but it's the same idea as the analysis for binary search. This is a big improvement over O(N)!
However, the problem with the new approach is that it assumes N is a power of 2. If N = 8, then we can easily have this:
 |
converts to |  |
But what if N is not a power of 2? If N = 9, then our whole approach falls apart. Now, N/2 is not an integer. So, what's our base case now? Before, we could stop when we reached N=1 since that was simple, but now we never reach 1 (or 0, another good base case). Additionally, non-integer exponentiation is best if hard, time-consuming, and often subject to rounding errors, so it's best to avoid it if possible. So, what do we do when the N is not a power of 2? We try to convert it into a power of 2!
Let's start with at N = 9:
Ok, let's try another: N = 14. What should we do this time? Following from N=9, you might try:
If something seems wrong with this, that's because we're starting to transform our divide and conquer algorithm into a non-divide-and-conquer algorithm. Using this approach to convert N into a power of 2 degrades the performance of our algorithm into a O(N) algorithm just like the earlier versions of integer exponentiation we looked at. Let's take a different approach to N = 14.
We've been dividing N by 2, so we aren't necessarily concerned that N is a power of 2, just that it's divisible by 2. So, if N isn't divisible (e.g. N=9), convert it into something that is divisible by 2. Now we have two cases:
This is what we did for N = 9 and it worked out. Let's look at how it works for N = 14:
|
Possible Implementation public long power(long X, long N) { if (N == 0) //base case return 1; else if (N == 1) //base case return X; else //recursive case { if (N % 2 == 0) { long result = power(X, N/2); //memoize return result * result; } else { long result = power(X, N/2); //memoize return result * result * X; } } } |
Trace of Recursive Calls
|
Code comparing this divide and conquer approach with the two we saw earlier can be found here: Power.java. The divide and conquer approach is in the method Pow3. Why do you think Pow3 has a base case of N == 0 (N.compareTo(zero) == 0)?
Now let's reconsider binary search, this time using using recursion with divide and conquer. Recall that the data must be in order already and we are searching for some object S.
Let's compare this recursive binary search to the iterative binary search (and to the sequential search). BinarySearchTest.java . Look at the number of comparisons needed for the searches. As N gets larger, the difference becomes very significant.
Read Chapter 18 of the Carrano text; it discusses both sequential search and binary search.
So far, every recursive algorithm we have seen can be done easily in an iterative way. Even the divide and conquer algorithms (Binary Search, Integer Exponentiation) have simple iterative solutions. Can we tell if a recursive algorithm can be easily done in an iterative way? Yes, any recursive algorithm that is exclusively tail recursive can be done simply using iteration without recursion. Most algorithms we have seen so far are exclusively tail recursive.
Tail recursion is a recursive algorithm in which the recursive call is the last statement in a call of the method. If you look back at the algorithms we've seen so far, this is generally true (ignore trace versions, which add extra statements). The integer exponentiation function does some math after the call, but it can still be done easily in an iterative way, even the divide and conquer version. In fact, any tail recursive algorithm can be converted into an iterative algorithm in a methodical way.
Why bother with identifying tail recursion and know that it's possible to convert to an iterative method? Recursive algorithms have overhead associated with them:
These are implementation details that algorithm analysis ignores. An iterative implementation of binary search and a recursive implementation will have the same runtime according to algorithm analysis. Recall that algorithms are independent of programming language, hardware, and implementation details. Deciding which algorithm to use and how to implement it are two separate, but very important, questions.
If recursive algorithms have this overhead, why bother with it? For some problems, a recursive approach is more natural and simpler to understand than an iterative approach. Once the algorithm is developed, if it is tail recursive, we can always convert it into a faster iterative version (ex: binary search, integer exponentiation). However, for some problems, it is very difficult to even conceive an iterative approach, especially if multiple recursive calls are required in the recursive solution. We'll take a look at some examples in the next section.
The idea behind backtracking is to proceed forward to a solution until it becomes apparent that no solution can be achieved along the current path. At that point, undo the solution (backtrack) to a point where we can proceed forward again and look for a solution.
In the 8 Queens Problem, you attempt to find an arrangement of queens on an 8x8 chessboard such that no queen can take any other in the next move. In chess, queens can move horizontally, vertically, or diagonally for multiple spaces.
How can we solve this with recursion and backtracking? All queens must be in different rows and different columns, so each row and each column must have exactly one queen when we are finished (why?). Complicating it a bit is the fact that queens can move diagonally. So, thinking recursively, we see that to place 8 queens on the board we need to:
Where does backtracking come in? Our initial choices may not lead to a solution; we need a way to undo a choice and try another one.
Using this approach, we have the JRQueens.java program. This program also creates a graphical display to show to recursive search for the solution. The method that does the recursive backtracking is trycol, reproduced below with the display-related statements removed:
The basic idea of the method is that each recursive call attempts to place a queen in a specific column. A loop is used, since there are 8 squares in the column. For a given method call, the state of the board from previous placements is known (i.e. where are the other queens?). This is used to determine if a square is legal or not. If a placement within the column does not lead to a solution, the queen is removed and moved down one row in that column. When all rows in a column have been tried, the call terminates and backtracks to the previous call (in the previous column). If a queen cannot be placed into column i, do not even try to place one onto column i+1; rather, backtrack to column i-1 and move the queen that had been placed there.
This solution is fairly elegant when done recursively. To solve it iteratively, it's rather difficult. We need to store a lot of state information (such as for each column so far, where has a queen been placed?) as we try (and un-try) many locations on the board. The run-time stack does this automatically for us via activation records. Without recursion, we would need to store / update this information ourselves. This can be done (using our own Stack rather than the run-time stack), but since the mechanism is already built into recursive programming, why not utilize it?
This is a simple puzzle and popular recursive algorithm problem. There are three rods. Two are empty and the third holds N disks. Each disk has a different diameter and they are stacked from widest to narrowest. Below is an image from Wikimedia Commons showing the starting setup:
The goal of the puzzle is to move the entire stack to another rod, obeying the following rules:
How do you solve such a puzzle?
A program solving the puzzle can be found at JRHanoi.java. Most of the code in the file is dedicated to displaying the solution. The key method is solveHanoi:
When a recursive algorithm has 2 calls, the execution trace is now a binary tree, as we saw with the solution to Tower of Hanoi. This execution is more difficult to do without recursion, but it is possible. To do it, programmer must create and maintain their own stack to keep all of the various data values. This increases the likelihood of errors / bugs in the code. Later, we will see some other classic recursive algorithms with multiple calls, such as MergeSort and QuickSort.
A very good demo of this can be found here: http://www.cs.cmu.edu/~cburch/survey/recurse/hanoiimpl.html.
Let's look at one more backtracking example: Finding words in a two-dimensional grid of letters. Given a grid and a word, is the word located somewhere within the grid? Each letter must touch the previous letter and we can only move right, left, up, and down. We can solve this with recursion. Let's take a look at how.
| w | a | r | g |
| b | c | s | s |
| a | a | t | s |
| t | r | y | x |
FindWord.java gives an implementation.
| << Previous Notes | Daily Schedule | Next Notes >> |
