Numerical Linear Algebra at the Institute for Numerical Analysis

A Journey Through the History of Numerical Linear Algebra by Claude Brezinski, Gérard Meurant, and Michela Redivo-Zaglia was published by SIAM in 2022. This comprehensive book chronicles the development of numerical methods for linear algebra problems, describes the relationship between these methods and computing tools in various time periods, and features 78 biographies of important researchers in the field. The following excerpt is adapted from chapter 5, which is called “Iterative Methods for Linear Systems.” It has been abridged and edited for clarity.

Here, we review the history of the Institute for Numerical Analysis (INA) at the U.S. National Bureau of Standards (NBS)—now the National Institute of Standards and Technology (NIST)—and describe its important role in the early development of numerical linear algebra. NBS was founded in 1901 as a physical laboratory to provide standard weights and measures. Edward Uhler Condon served as NBS director from 1945 to 1951, while John Hamilton Curtiss joined in 1946 as an assistant to the director and took charge of statistical problems. That same year, Condon instructed Curtiss to assess the federal need for computers and the potential for a national computing center. This investigation had its source at the Office of Naval Research (ONR), as ONR’s Mathematics Branch was formed in 1946 with Mina Rees as its head. It was suggested that NBS and ONR jointly establish a computing center to develop and use computers.

Curtiss quickly realized that the necessary mathematics to exploit the new computers also had to be developed, and he and Rees were largely responsible for the formation of INA. In the fall of 1946, Curtiss traveled the West Coast to seek a home for the proposed INA. Although INA was ultimately established at the University of California, Los Angeles (UCLA), the Institute was originally housed at NBS in Washington, D.C.

INA’s main goal was to plan and conduct a program of research in pure and applied mathematics that ultimately developed methods of analysis to enable the most efficient and general use of high-speed automatic digital computers. In response to some problems with the procurement of computers, a proposal in early 1948 recommended that Harry Douglas Huskey develop a small machine at INA; his efforts led to the Standards Western Automatic Computer (SWAC) [4]. The project began in January 1949, and the first Williams tube machine to be completed in the U.S. was dedicated on April 7, 1950. After a rather long period of debugging, SWAC finally became a reliable machine with many significant accomplishments (see Figure 1). When INA operations ceased in 1954, SWAC was transferred to UCLA and continued to function until 1967.

The first INA appointees were Olga Taussky, her husband John Todd, and Albert Cahn, Jr. The Institute’s initial two projects were titled Characteristic Roots of Matrices and Applications of Automatic Digital Computing Machines in Algebra and Number Theory. The inaugural report about the activities of the National Applied Mathematics Laboratories (NAML)—of which Curtiss was named chief in July 1947—was published in January 1948; NAML was later renamed the Applied Mathematics Division (AMD). The new INA building at UCLA opened in April 1948; Gertrude Blanch organized the computational unit at that location. The months of April, May, and June 1948 were an organizational period during which scientific and computational staff were hired.

Douglas Rayner Hartree, a mathematical physicist from Cambridge, U.K., became INA’s first director. He stayed only a short time, however, after which Curtiss served as acting chief until the fall of 1949. George Elmer Forsythe was INA’s first regular member of research staff, Cornelius Lanczos was the second, and Wolfgang Richard Wasow was the third. Magnus Rudolph Hestenes, a professor at UCLA, was also heavily involved with INA. He assumed the titles of Assistant Director of Research and UCLA Liaison Officer in the mid-1950s.

In September 1949, John Barkley Rosser was appointed as INA’s new director. Despite his brief one-year term, he had a great impact on INA. Rosser organized a weekly seminar that focused on iterative methods for the solution of linear equations and the computation of eigenvalues, while Forsythe sought to classify the various known methods for solving linear systems. Hestenes, William Karush, and Marvin Leonard Stein studied methods for computing eigenvalues, with a particular emphasis on gradient, power, and inverse power methods. Lanczos worked on Chebyshev polynomials and orthogonalization techniques for computing eigenvalues and solving linear systems.

After Rosser returned to Cornell University in September 1950, Fritz John—an expert in partial differential equations—became INA director for the subsequent year. From July to September 1950, staff included John, Blanch, Forsythe, Huskey, Karush, Lanczos, Wasow, Paul Erdős, Richard Feynman, Mark Kac, Edward James McShane, William Edmund Milne, Theodore Samuel Motzkin, and Otto Szász. 

1951 was an important year for INA. That summer, Hestenes devised the conjugate gradient method for the solution of linear systems. Theoretically, this method delivers the solution in at most \(n\) iterations for a matrix of order \(n\). Hestenes offered three versions of his routine and later recounted the details in a NIST report with Todd [3]:

“When [Eduard Ludwig] Stiefel arrived at INA from Switzerland, the librarian gave him a paper describing this routine. Shortly thereafter, Stiefel came to Hestenes’ office with this paper in hand and said, “Look! This is my talk.” It turned out that Stiefel had invented the same algorithm from a different point of view. He looked upon it as a relaxation routine, whereas Hestenes viewed it as a gradient routine on conjugate subspaces. The term ‘conjugate gradient’ was coined by Hestenes.”

Stiefel accepted a joint appointment with INA and UCLA so that he and Hestenes could write an extensive paper about the conjugate gradient method. Stein, Robert Mayo Hayes, Urs Hochstrasser, and W. Wilson carried out the numerical experiments — although Forsythe, Karush, Rosser, and Lowell J. Paige also deserve credit for developing the conjugate gradient routine. In fact, Rosser and Stiefel presented the conjugate gradient method at the Symposium on Simultaneous Linear Equations and the Determination of Eigenvalues in August 1951, which was a part of NBS’ Semicentennial Celebration.

1953 saw the onset of problems at NBS that were linked to the so-called “battery additive scandal.” Beginning in the early 1920s, NBS tested many commercial battery additives and never found any to be beneficial. The owner of a small company in California proclaimed that his product—a chemical additive called AD-X2—had a special composition and advantageous effect. The company lobbied at Congress and encouraged satisfied users to flood Congress and NBS with testimonial letters. NBS performed further tests, with the same results as before. Due to pressure on the federal administration, Allen Varley Astin (then-director of NBS) was asked to resign by the Assistant Secretary of Commerce for Domestic Affairs.

This situation generated unfortunate consequences for INA, as the Department of Defense (DoD) created new rules that complicated funding arrangements with ONR. Although Astin was reinstated as director in August 1953, four sections of NBS were transferred to DoD. One investigation committee praised AMD’s activities but questioned the appropriateness of INA’s inclusion within NBS. Eventually, NBS was forced to relinquish its sponsorship of INA.

On June 30, 1954, INA held a “Gala Party and Luncheon” in honor of its impending transfer to UCLA, the people leaving INA, and the individuals staying at UCLA. This marked the end of an Institute that produced many interesting results during its brief existence. It was a pity—and probably detrimental to U.S. research—that such a fascinating place was shuttered due to administrative problems.

The main sources of information about INA’s history are the 1991 report by Hestenes and Todd [3] and the quarterly reports of NAML and AMD activities. Additional information about NBS [1] is also available in papers by Todd [8, 9], Gene Golub [2], and Dianne Prost O’Leary [5-7].

Enjoy this passage? Visit the SIAM Bookstore to learn more about A Journey Through the History of Numerical Linear Algebra and browse other SIAM titles. Additionally, a book review of this text by Volker Schulz of Trier University recently published in the June 2024 issue of SIAM Review (Vol. 66, Issue 2).

References
[1] Boisvert, R.F., Donahue, M.J., Lozier, D.W., McMichael, R., & Rust, B.W. (2001). Mathematics and measurement. J. Res. Natl. Inst. Stand. Technol., 106(1), 293-313.
[2] Golub, G.H., & O’Leary, D.P. (1989). Some history of the conjugate gradient and Lanczos algorithms: 1948-1976. SIAM Rev., 31(1), 50-102.
[3] Hestenes, M.R., & Todd, J. (1991). NBS-INA – The Institute for Numerical Analysis – UCLA 1947-1954. NIST Special Publication 730. Gaithersburg, MD: National Institute of Standards and Technology.
[4] Huskey, H.D. (1950). Characteristics of the Institute for Numerical Analysis computer. Math. Tables Other Aids Comp., 4(30), 103-108.
[5] O’Leary, D.P. (1996). Conjugate gradients and related KMP algorithms: The beginnings. In L. Adams & J.L. Nazareth (Eds.), Linear and nonlinear conjugate gradient-related methods (pp. 1-8). Philadelphia, PA: Society for Industrial and Applied Mathematics.
[6] O’Leary, D.P. (2001). Commentary on an iteration method for the solution of the eigenvalue problem of linear differential and integral operators by Cornelius Lanczos. In D.R. Lide (Ed.), A century of excellence in measurements, standards, and technology: A chronicle of selected NBS/NIST publications 1901-2000 (pp. 77-80). NIST Special Publication 958. Gaithersburg, MD: National Institute of Standards and Technology.
[7] O’Leary, D.P. (2001). Commentary on methods of conjugate gradients for solving linear systems by Magnus R. Hestenes and Eduard Stiefel. In D.R. Lide (Ed.), A century of excellence in measurements, standards, and technology: A chronicle of selected NBS/NIST publications 1901-2000 (pp. 81-85). NIST Special Publication 958. Gaithersburg, MD: National Institute of Standards and Technology.
[8] Todd, J. (1975). Numerical analysis at the National Bureau of Standards. SIAM Rev., 17(2), 361-370.
[9] Todd, J. (1990). The prehistory and early history of computation at the U.S. National Bureau of Standards. In S.G. Nash (Ed.), A history of scientific computing (pp. 251-268). New York, NY: Association for Computing Machinery.