Article
How Yelp and Angie’s List revolutionized supplier performance evaluation in government
As a kid growing up in the 1980s, I watched a lot of television. While the sitcoms were memorable, for me, it was the commercials that I loved. And my favorite one of all was the Reese’s Peanut Butter Cups ad simply called “Walkers.” An oblivious young man struts down a sidewalk, holding a big candy bar and says, “mmm, chocolate.” Rapidly approaching him, listening to a Sony Walkman is an equally oblivious young woman eating peanut butter out of a jar, exclaiming, “mmm, mmm Pea-NUT butter!” Predictably, they collide on the sidewalk and utter the most famous advertising lines of the decade, “You got your chocolate in my peanut butter. You got your peanut butter in my chocolate.” Of course, the viewers at home know that when you put these seemingly disparate products together, the result is confectionary perfection.
You may be asking what this has to do with government procurement? Just as a candy bar and a jar of peanut butter are an unlikely, but ultimately delightful combination, so too are consumer technology and government procurement processes. At first, they may appear to be hopelessly at odds with one another, but when forged together, they are creating a revolution in the way governments select and manage their suppliers. In this article, I will talk about how modern tech and old school procurement have come together, but first some history.
As the chief procurement officer of Pennsylvania, one of the greatest challenges I faced was evaluating the past performance of suppliers who had responded to the Commonwealth’s solicitations. We believed that understanding how a supplier performed for similar governments was vitally important. After all, the best predictor of a supplier’s future performance for us was their past performance for other customers. But the Commonwealth, like most governments, had a process that did little to give us insights about a supplier’s past performance.
In our RFP process, we asked the supplier for three references. Knowing that we would be giving points based on how well the reference spoke about them, the suppliers had little choice but to send us the three very best references they could find. While on rare occasions, a reference told us something critical about the supplier, the vast majority of references were exceedingly positive and taught us little. In talking with my procurement colleagues across the country, I heard that some evaluation committees stopped calling references, but instead literally “checked the box” and said they did. And it’s hard to blame them, since they knew the likely outcome of the call.
It should come as no surprise to us that great suppliers do great work for customers over and over again. We can predict that a supplier who has consistently exceeded the service level agreements for other customers will perform ably for us. This is especially true if their stellar work was for customer of similar size and complexity and if the work performed was very similar in scope. Conversely, we are right to assume that a supplier whose performance history was littered with major issues, contract disputes, cure letters and terminations is more likely than not going to face performance challenges for us in a new contract. And while these trends are utterly predictable, for decades, they have not been unavoidable. And the consequences have been grave.
A state department of corrections hired a general contractor to build a new, state of the art prison several years ago. Nearly from the start of the project, it was plagued with problems, many of which were caused by the GC. The results were ugly. Two years late in opening, tens of millions of dollars in fines, thousands of construction deficiencies. One would think that this contractor would never be awarded a contract by this state again, given all the problems. But just a few years later, another state agency awarded it a multi-million-dollar contract. And predictably, tragically, major problems ensued early in the project.
Procurement officials know all too well that while they rarely (and sadly) get credit when things go right, they are the first to be blamed when things go wrong. Having led procurement teams, I experienced this myself and it always stings. But in this example, we can’t fault the procurement professionals. When the second agency conducted its procurement, it followed the state’s process and asked for the standard three references. Of course, the contractor didn’t provide the state corrections department, just down the road, because they knew they’d be a negative reference. So the procurement staff could only contact the references provided and make an award based on the official references. They had no way of knowing that their sister agency had had problems with the same vendor.
This story is unfortunately repeated every day across the United States. And just think – if two procurement professionals working for the same government can’t find out how a prospective supplier performed for one before making an award, imagine how difficult it must be to find out about an RFP finalist performed for governments in different states or time zones. This vitally important task – learning about a supplier’s real past performance – has seemed utterly impossible to many of us for years. But just like the young man holding the candy bar, a solution was walking right at us. Only we didn’t know it.
In 1979, as I was absorbing commercials in my parents’ house, Tim and Nina Zagat were starting a revolution that we now know as crowdsourcing. The Zagats loved restaurants. They had a close group of friends who shared this passion. They decided that rather than relying on the lengthy reviews written by newspaper food critics (the equivalent to vendor supplied references) they would create a survey about restaurants in New York, have their friends respond and then aggregate those ratings in a thin maroon book they published called the Zagat Survey. The books flew off the shelves of bookstores as diner across New York and eventually the world realized that thousands of fellow diners could give them better advice than two or three food critics.
Separated by fifteen years and 500 miles, a young woman named Angie Hicks used the power of crowdsourcing to solve another challenge, home repair. Angie’s boss, Bill Oesterle had just moved to Columbus, Ohio and struggled to find good contractors. He had previously belonged to a group of neighbors in Indiana who shared information about just about everything and he wanted to recreate it in Columbus. He tasked Angie with building a group of homeowners who could inform each other about contractors they had used – good and bad. Angie went door-to-door, literally, talking to anyone who would answer. Eventually, her hard work paid off and she built a list of neighbors who would collaborate together. And it wasn’t just any list. It was Angie’s List.
The peer reviewed crowdsourcing revolution, begun by Tim and Nina Zagat, Bill Oesterle and Angie Hicks, became the mainstay in the American economy by a twenty-something programmer in San Francisco named Jeremy Stoppelman. In 2004, Stoppelman, a former PayPal engineer was felled by the flu and was frustrated that he couldn’t find a doctor on the internet. That frustration led to the creation of Yelp which has revolutionized the way that people across the world find the best local businesses. Today, 178 million people across the world go to Yelp before choosing where to eat, shop, work out or conduct other business.
In our personal lives, we as consumers use online ratings and review platforms to help us make the best possible decisions. When we plan trips, we read reviews from thousands of fellow travelers on TripAdvisor before we book a hotel. If we are looking for a new job, we read reviews from dozens of employees on Glassdoor about what it’s really like to work there. And of course, we turn to Yelp to hear from fellow diners about the restaurants where we are thinking about eating.
The crowdsourcing revolution that brought us Yelp, TripAdvisor and Glassdoor is not just on its way to public procurement. It’s here. Last summer, the Commonwealth of Pennsylvania launched a first-in-the-nation partnership with the ratings and review platform Procurated to help purchasers make better decisions based on peer reviews. Since the early 2000s, Pennsylvania’s Department of General Services has established contracts for use by local governments, school districts, universities and non-profits through its COSTARS program. In less than a year, roughly 10,000 reviews have been written about 1500 suppliers. Just as you can look up a restaurant and read reviews on Yelp, governments in Pennsylvania regularly login to Procurated to see what their colleagues across the state are saying about a supplier before they make an award or place an order with them.
While a crowdsourcing tool can be very effective in spreading the word about poor performing suppliers, like the contractor referenced earlier, it is equally effective in sharing stories of successful supplier partnerships. As the COVID-19 crisis struck, procurement professionals across the country desperately searched for suppliers who could provide them with hard-to-find personal protective equipment. Thanks to the power of crowdsourcing tools like Procurated, procurement professionals were able to tell each other about companies that really came through for them, furnishing them with elusive products at reasonable prices, ultimately saving the lives of the medical professionals on the front lines of the crisis. This interstate collaboration will remain vital as our nation continues to battle the coronavirus crisis.
And as our thoughts turn from coronavirus response to the national discussion on inclusion as Americans protested the death of George Floyd, the power of crowdsourcing tools will again manifest itself. Governments across the country will redouble efforts to make the contracting process more inclusive and will try to expand opportunities for minority owned firms. The task of turning that goal into results will fall to procurement officers at the state and local level. On the Procurated platform, purchasers can find minority owned businesses in their communities and solicit them to perform work for their governments. And as they look at contracting with these firms, they can read the positive reviews written by their fellow procurement directors.
The power of this performance data doesn’t stop when a new contract is signed. Crowdsourced performance feedback enables government procurement leaders to have data-driven performance conversations with their existing suppliers. Many procurement teams struggle with vendor performance management because they don’t have the tools, technology, or time to facilitate those conversations in a meaningful way. The Procurated platform empowers contract managers to hold vendors accountable and work together to elevate performance.
Government procurement is often labeled, unfairly in my opinion, as being a laggard when it comes to adopting new technology. But by embracing the cutting edge, crowd-sourcing technology used by consumers, it is revolutionizing the way it evaluates vendor past performance. In so doing, we are radically improving the quality of suppliers we are selecting and the service they provide to our citizens.
David Yarkin is the former Chief Procurement Officer of the Commonwealth of Pennsylvania and a nationally recognized expert on procurement innovation in the public sector. David is the CEO of Procurated, a supplier ratings and review platform for government and the President of Government Sourcing Solutions, a supply chain consulting firm that has worked with 34 states and many large municipalities since its founding 15 years ago.
