Worst-case analyses are generally easier than average-case analyses. The reason is that, for worst-case analyses, one simply needs to give an upper bound on how much effort is required and then exhibit a specific example that attains this bound. There are two serious difficulties here. The first is that it is not clear at all how one should model the space of random problems. Secondly, given such a model, one must be able to evaluate the amount of effort required to solve every problem in the sample space. Therefore, worst-case analysis is more tractable than average-case analysis, but it is also less relevant to a person who needs to solve real problems.
In this chapter, we shall give a worst-case analysis of the simplex method. Later, in Chapter 12, we shall present results of empirical studies that indicate the average behavior over finite sets of real problems. Such studies act as a surrogate for a true average-case analysis. Measuring the Size of a Problem Before looking at worst cases, we must discuss two issues.
First, how do we specify the size of a problem? Two parameters come naturally to mind: m and n.
Linear Optimization: The Simplex Workbook
However, we should mention some drawbacks associated with this choice. First of all, it would be preferable to use only one number to indicate size. Since the data for a problem consist of the constraint coefficients together with the right-hand side and objective function coefficients, perhaps we should use the total number of data elements, which is roughly mn. Efficient implementations do indeed take advantage of the presence of lots of zeros, and so an analysis should also account for this.
Hence, a good measure might be simply the number of nonzero data elements. This would definitely be an improvement, but one can go further. On a computer, floating-point numbers are all the same size and can be multiplied in the same amount of time. But if a person is to solve a problem by hand or use unlimited precision computation on a computer , then certainly multiplying 23 by 7 is a lot easier than multiplying This measure is popular among most computer scientists and is usually denoted by L. However, with a little further abstraction, the size of the data, L, is seen to be ambiguous.
As we saw in Chapter 1, real-world problems, while generally large and sparse, usually can be described quite simply and involve only a small amount of true input data that gets greatly expanded when setting the problem up with a constraint matrix, right-hand side, and objective function. So should L represent the number of bits needed to specify the nonzero constraint coefficients, objective coefficients, and right-hand sides, or should it be the number of bits in the original data set plus the number of bits in the description of how this data represents a linear programming problem?
No one currently uses this last notion of problem size, but it seems fairly reasonable that they should or at least that they should seriously consider it. Anyway, our purpose here is merely to mention that these important issues are lurking about, but, as stated above, we shall simply focus on m and n to characterize the size of a problem. Measuring the Effort to Solve a Problem The second issue to discuss is how one should measure the amount of work required to solve a problem.
Unfortunately, there are hopefully many readers of this text, not all of whom use the exact same computer. Even if they did, computer technology changes rapidly, and a few years down the road everyone would be using something entirely different. So the time needed to solve a problem, while the most desirable measure, is not the most practical one here. Fortunately, there is a fairly reasonable substitute.
Algorithms are generally iterative processes, and the time to solve a problem can be factored into the number of iterations required to solve the problem times the amount of time required to do each iteration. The first factor, the number of iterations, does not depend on the computer and so is a reasonable surrogate for the actual time. This surrogate is useful when comparing various algorithms within the same general class of algorithms, in which the time per iteration can be expected to be about the same among the algorithms; however, it becomes meaningless when one wishes to compare two entirely different algorithms.
14 editions of this work
For now, we shall measure the amount of effort to solve a linear programming problem by counting the number of iterations needed to solve it. Well, we have already seen that for some pivoting rules it can cycle, and hence the worst-case solution time for such variants is infinite. However, what about noncycling variants of the simplex method? And how big is it? It should be noted that, even though typographically compact, the expression 2n is huge even when n is not very big.
UC Davis Mathematics :: Syllabus Detail
We shall now give an example, first discovered by V. Klee and G.
The example is quite simple to state: maximize 4. It is instructive to look more closely at the constraints. The first constraint simply says that x1 is no bigger than one. With this in mind, the second constraint says that x2 has an upper bound of about , depending on how big x1 is.
Similarly, the third constraint says that x3 is roughly no bigger than 10, again, this statement needs some adjustment depending on the sizes of x1 and x2. For this reason, the feasible region for the Klee—Minty problem is often referred to as the Klee—Minty cube.
An n-dimensional hypercube has 2n vertices, and, as we shall see, the simplex method with the largest-coefficient rule will start at one of these vertices and visit every vertex before finally finding the optimal solution. The right-hand sides still grow by huge amounts as i increases. Finally, we wish to add a constant to the objective function so that the Klee—Minty problem can finally be written as maximize 4. In Exercise 4. Using the largest coefficient rule, the entering variable is x1. From the fact that each subsequent bi is huge compared with its predecessor it follows that w1 is the leaving variable.
A few observations should be made. First, every pivot is the swap of an xj with the corresponding wj. Also note that the final dictionary could have been reached from the initial dictionary in just one pivot if we had selected x3 to be the entering variable. But the largestcoefficient rule dictated selecting x1. It is natural to wonder whether the largestcoefficient rule could be replaced by some other pivot rule for which the worst-case behavior would be much better than the 2n behavior of the largest-coefficient rule.
So far no one has found such a pivot rule. However, no one has proved that such a rule does not exist either. Finally, we mention that one desirable property of an algorithm is that it be scale invariant. This means that should the units in which one measures the decision variables in a problem be changed, the algorithm would still behave in exactly the same manner. The simplex method with the largest-coefficient rule is not scale invariant.
Now, the largest-coefficient rule picks variable x3 to enter. Variable w3 leaves, and the method steps to the optimal solution in just one iteration. There exist pivot rules for the simplex method that are scale invariant.
- The Leadership Experience;
- The Democracy Of God: An American Catholicism?
- Foundations and Extensions?
- Parallel Computing on Heterogeneous Networks;
- Department of Mathematics Syllabus.
But Klee—Minty-like examples have been found for most proposed alternative pivot rules whether scale invariant or not. In fact, it is an open question whether there exist pivot rules for which one can prove that no problem instance requires an exponential number of iterations as a function of m or n. Fix k and consider the pivot in which xk enters the basis and wk leaves the basis. Show that the resulting dictionary is of the same form as before.
Show that by ignoring feasibility preservation of intermediate dictionaries this dictionary can be arrived at in exactly k pivots. Hint: see Exercise 2. For a survey of probabilistic methods, the reader should consult Borgwardt b. For many years it was unknown whether linear programming had polynomial complexity. The Klee—Minty examples 54 4. In , Khachian gave a new algorithm for linear programming, called the ellipsoid method, which is polynomial and therefore established once and for all that linear programming has polynomial complexity.
The collection of all problem classes having polynomial complexity is usually denoted by P. An important problem in theoretical computer science is to determine whether or not P is a strict subset of N P. The study of how difficult it is to solve a class of problems is called complexity theory.
The dual of this dual linear program is the original linear program which is then referred to as the primal linear program. It turns out that every feasible solution for one of these two linear programs gives a bound on the optimal objective function value for the other. These ideas are important and form a subject called duality theory, which is the topic we shall study in this chapter. But how good is this bound?
Is it close to the optimal value? To answer, we need to give upper bounds, which we can find as follows. We have localized the search to somewhere between 9 and These bounds leave a gap within which the optimal solution lies , but they are better than nothing. Furthermore, they can be improved.
To get a better upper bound, we again apply the same upper bounding technique, but we replace the specific numbers we used before with variables and then try to find the values of those variables that give us the best upper bound. So we start by multiplying the two constraints by nonnegative numbers, y1 and y2 , respectively. The fact that these numbers are nonnegative implies that they preserve the direction of the inequalities.
This problem is called the dual linear programming problem associated with the given linear programming problem. In the next section, we will define the dual linear programming problem in general.
- Remaking Reality: Nature at the Millenium?
- 48665fab8ee2410d941339d0fae00e47c088.pdf - Linear....
- Stanford Libraries?
The Dual Problem Given a linear programming problem in standard form, maximize 5. Since we started with 5. Our first order of business is to show that taking the dual of the dual returns us to the primal. To see this, we first must write the dual problem in standard form. That is, we must change the minimization into a maximization and we must change the first set of greater-thanor-equal-to constraints into less-than-or-equal-to. Of course, we must effect these changes without altering the problem. The Weak Duality Theorem As we saw in our example, the dual problem provides upper bounds for the primal objective function value.