Some Recollections of Trefethen and Bau on the Occasion of Its 25th Anniversary

When I was a graduate student at Stanford in 1978, I remember Gene Golub telling me that Pete Stewart’s Introduction to Matrix Computations was very good (he was right). Of course, Gene added, it’s quite a few years old. But it had only been published five years before, in 1973! So it almost seems like science fiction that Numerical Linear Algebra—which was published in 1997—is still going strong today, 25 years later.

I was not a numerical linear algebraist by training; I’d written my thesis with Joe Oliger on finite difference methods for partial differential equations. Still, I’d been exposed to the subject in courses by Golub, Jim Wilkinson, and Cleve Moler — and indeed by the whole atmosphere of Golub’s numerical analysis group at Stanford. When I landed at MIT as the only numerical analyst on the faculty, naturally enough they wanted me to teach this subject. We put together a course numbered 18.335, and the masterpiece Matrix Computations by Golub and Van Loan had just come out. I had the perfect combination of first-rate students from a mix of departments with an outstanding reference text that provided details about everything I could possibly want to teach. MATLAB emerged around the same time and changed my outlook permanently, making me more professionally interested in linear algebra and also a believer in the importance of programming at a high conceptual level.

The Golub and Van Loan book, however, was too much. After a few years of teaching numerical linear algebra, I had developed my own views on what subset of topics should be included in a course and in which order they should appear to avoid boring the students with Gaussian elimination in lecture 1. By this point I was at Cornell and 18.335 had become Applied Math 621 (Charlie Van Loan was now my colleague, but he generously allowed me to teach his subject anyway). I found myself tempted to write a book of my own that would be shorter and more student-friendly than Golub and Van Loan, and at just the right moment, along came a Ph.D. student who liked to write and with whom I got along very well. Most readers have probably not met David Bau, but that’s your loss. He’s an extraordinary person and we have a lot in common, including growing up in a western suburb of Boston and attending Phillips Exeter Academy and Harvard. Now that my course had been running for a few years and such a simpatico student had arrived at Cornell, I had an idea: I could ask David to take thorough notes of my lectures and we could turn them into a book. I think the end result benefitted from his freshness as a newcomer to the subject, and indeed from my own fresh perspective as a nonexpert.

It all unfolded as I had hoped, and David contributed hugely to Numerical Linear Algebra — including the first sentence, which I like very much: “You already know the formula for matrix-vector multiplication.” The figures are drawn straight in native PostScript, with no MATLAB or other intermediary. Somewhere along the way, David had become a PostScript hacker. At SIAM, Vickie Kearn and Beth Gallagher made the book attractive and free of glitches.

I have happy memories of teaching numerical linear algebra at Cornell as these lectures took shape. One year saw about 25 participants in the course, all of whom were Ph.D. students except for one undergraduate. The undergraduate turned in dazzlingly good homework assignments and got the highest mark in the class. He was called Jon Kleinberg.

I like a lot of things about Trefethen and Bau, including the short chapters — a pattern I’ve followed in my subsequent books. Perhaps my single favorite item is the observation that for computing the QR decomposition, Gram-Schimdt is triangular orthogonalization whereas Householder is orthogonal triangularization. I also enjoy the section “When Vectors Become Continuous Functions,” which led to Chebfun and a whole new way of thinking in terms of continuous analogues of all the classical structures and algorithms.

Our book came along at the perfect time. Classical ideas had been established, including Krylov subspace iterations, yet it was still an era when many mathematical scientists were barely aware of the singular value decomposition (SVD). This may have been the first textbook to put the SVD up front as a fundamental topic, not just a subordinate tool to another problem like least squares. Page 26 provides a sketch emphasizing that every \(m \times n\) matrix maps the unit sphere to a hyperellipse, without exception. Numerical Linear Algebra seems to have been welcomed as a good foundation for all kinds of things, and Yuji Nakatsukasa’s afterword for the 25th anniversary edition is an exciting review of developments in the years after its initial publication.

With each passing decade since the introduction of computers, linear algebra has become more important. This is a trend that I don’t think Alan Turing and John von Neumann saw coming, yet has had something of the inexorability of Moore’s Law (if not quite the same time constant). The growth is continuing in the present era of data science, as everybody in the tech world now seems to want to know more linear algebra. Back in 1997, we hoped the SVD would come to be appreciated as important; we hardly imagined it would one day even be cool.

The 25th anniversary edition of Numerical Linear Algebra will be available for purchase at the 2022 SIAM Annual Meeting, which will take place in a hybrid format from July 11-15, 2022, in Pittsburgh, Pa. A book signing with Nick Trefethen will accompany its debut.