The Saga of PLS


by Gaston Sanchez

You’ve got to be kidding me

The first time I heard about PLS was in October 2004 when I first met professor Tomàs Aluja—my former PhD adviser. I had just recently moved to Barcelona to start my graduate studies in statistics, and I had been designated Tomàs’s new mentee. At the end of our first meeting when I was about to leave his office, Tomàs asked me a seemingly intrascendental question: “Have you ever heard about PLS?”

“P. L. S.?”

I repeated those letters to myself without pronouncing any words. Fearing to give a bad impression to my brand new mentor, I quickly scanned the hard-disk of my brain in search for any hint. Unsuccessfully struggling to find any answer, I ended up with a perfect clueless expression in my face.

I had no idea what PLS was or what it meant.

The reason why Tomàs asked me about PLS was because he was in charge of organizing PLS’05 - The 4th International Symposium on Partial Least Squares and Related Methods for the following year (September, 2005). He wanted to know if I would be willing to help him with some of the organizational details for the symposium. Little did I know the tremendous impact those three mysterious letters would have in the coming years of my analytics career.

I became involved with PLS in a very unintended way. One of my first duties as part of the PLS’05 staff, was being in charge of receiving the abstracts and paper submissions for the symposium, making sure they complied with the strict guidelines-for-authors. I spent my time sending emails and pestering authors kindly requesting them to fix the page layout, change the font size, reduce the number of pages, and things like that. One of the orders Tomàs gave me was to read all the papers. Grudgingly, I followed his command and sucked it all up. However, reading doesn’t necessarily mean understanding. From all the submitted papers, I was able to understand three of them… at the most. And even then my understanding was very superficial. The common thing about those papers was their very applied and digestable examples that prevented me from getting lost with all the unintelligible details. By the way, this is NOT how anyone should ever try to learn about PLS methods.

I had a more formal and intensive introduction to the PLS framework with Tomàs’s course “Advanced Methods for Multivariate Data Analysis.” The outline of the course was based on the great book La Régression PLS: Théorie et Pratique by Michel (Tenenhaus, 1998). The book is a marvelous piece, with Michel’s unique eloquent writing style, explaining things in a very detailed way, using a clear notation, and full of examples and graphics. I guess everything would have been fine if it wasn’t for the fact that I had a very limited knowledge of French. Which it only makes it harder when you’re not only a PLS novice but also a French-language beginner. Luckily, google translate was already available at that time, providing a considerable help in deciphering some of the book’s content.

Tomàs’s class was a very hands-on course. Each student was responsible of studying a chapter from the book, programming the corresponding method in R, and giving a lecture to the rest of the class. I got to present chapter 8—PLS Regression for one response variable. Tomàs kept for himself the presentation of the PLS Path Modeling (PLS-PM) approach for the last lecture. That day he introduced the technique with an example using the famous Russett dataset, and then he proceeded to talk about how the method worked. When he got to the description of the PLS-PM algorithm, however, I was having a hard time understanding all the steps. The worst part was my fruitless attempt trying to connect the PLS regression algorithms with the PLS-PM algorithm. The question that was driving me crazy was: Where on earth were the PLS regressions in the PLS-PM algorithm? I raised my hand and asked Tomàs if he could please show the rest of the class where the PLS regressions were. To my surprise, his answer was even more disconcerting: “Regressions? What regressions? This has nothing to do with PLS Regression. This is PLS Path Modeling.” I’m pretty sure Tomàs does not even remember these details. But they definitely had a profound impact on me.

All I remember thinking was something like: “You’ve got to be kidding me, this PLS stuff makes no sense whatsoever!” I was definitely biting off more than I was able to chew.

I won’t lie to you. My introduction to the PLS world was not a smooth one. Besides my very limited knowledge of French, another difficulty was my equally limited knowledge of the techniques in Michel’s book like Canonical Correlation Analysis, Redundancy Analysis, and Inter-Battery Factor Analysis. One of the morals from this experience was that I needed to learn French; the other moral was that this was also not the best way to start learning about PLS from scratch.

I felt so confused and disoriented, but eventually I managed to my find way out through the PLS labyrinth over the following years. My biggest consolation, to my suprise, was when I heard Michel Tenenhaus story in front of the audience at one of the PLS Research Workshops in Paris, 2009. He told us about his PLS tale: having started studying PLS regression, he was later referred to PLS Path Modeling, only to find out that, initially, nothing made sense to him. I was so relieved in knowing that I was not the only one with the same big confussion. Even a PLS jedi master like Michel had faced the same puzzle I had struggled with.

I became obsessed with the story behind PLS since the very first moment I started to work on my PhD dissertation about PLS Path Modeling. As a grad student in the fall of 2005, I remember googling about “Partial Least Squares” and getting links back that talked about PLS Regression. When I tried to be more specific and searched for “Partial Least Squares Path Modeling,” there was nothing but again PLS Regression retrieved results only. Why was there almost nothing about PLS Path Modeling? There were virtually no resources about the original works by Herman Wold, or the so-called Structural Equation Models via PLS approach. Moreover, most of the available resources on the Web were basically chemometrics-related material. It is amazing to see how much things have changed in just a decade. But it hasn’t been easy. It has required a titanic effort coordinating the work of a large number of scholars. Gradually, the barriers have been pushed in order to revive a methodological framework that was heading toward the annals of statistical methods that sadly end up in the dead archives of university libraries.

One of the very first things I knew I wanted to do was to find out more about the history of PLS methods. The main resource I had in my hands was a tutorial on PLS Path Modeling, which was basically a draft for the famous article “PLS Path Modeling” by Tenenhaus et al (2005). I started tracking the references listed in the tutorial, and I was happy to find that the Economics and Math libraries of the University of Barcelona had copies of the two volumes Systems under indirect observation: Causality, structure, prediction, the intimidating treatise Latent Variable Path Modeling with Partial Least Squares (Lohmoller, 1989), the Encyclopedia of Statistical Sciences with the entry of PLS (Wold, 1985a), and The Making of Statisticians containing the autobiographical essay of Herman (Wold, 1982a). Little by little I started to dig out and uncover fragments of a fascinating story.

All I wanted was to be able to answer the who, when, how, and why. Who created PLS? When was it developed? How was the development process? And why? How does something such as PLS is developed? What were the circumstances that gave birth to that framework? In this sense, the quest became a quest not only about PLS itself but also about its creator, about his mind, his ideas, and surrounding context. Among all my inquires, one question has remained for years in the back burner of my head: Why was PLS Path Modeling not that well known? Think about it. By 2005, the first works about PLS had already been present for almost 30 years! Yet, having access to those references was—and still is—ridiculously difficult. I think I have now most of the elements to answer the questions that have been lingering since that fall of 2005.

Next chapter