Forward Thinkers, Strategy

Elevating Analytics: Jason Thompson & 33 Sticks

Jason Thompson and I share many things in common. We were born a few days apart. We recall spending too much time playing games on Apple IIe computers. We have an almost spiritual response to most of the Coen brothers’ movies. And we have been collaborating professionally for about ten years. I traveled to Utah to learn more about how 33 Sticks, the Analytics boutique he founded with Hila Dahan, inspires customers to work differently.

Jason and Hila founded 33 Sticks about seven years ago and I’ve experienced first-hand the key principles that define 33 Sticks’ competitive advantage: the company employs a handful of rock star employees who typically work (very) remotely with clients and religiously abstain from billing by the hour. Sounds simple…but speak with any 33 Sticks customer and you will hear something more…a deeper connection to the 33 Sticks team where personal stories are shared freely and relationships span multiple companies.

As I drive the 30 minutes from Salt Lake City to Jason’s home near American Fork, the landscape gently reminds me that I am traveling a world away from my home outside of Boston. Mountains are always present in this part of the country, and distance is faithfully measured relative to the Salt Lake Temple downtown. As I drive farther from Salt Lake, Google Maps struggles to finish announcing intersections before I pass through them. But then I reach a point where the mountains seem to take over, streets have names, and horses linger in open grass fields between manicured subdivisions. Tech is growing here at a torrid pace, and traffic is starting to grind along the wide roads. I arrive at Jason’s home, the closest thing to a headquarters for 33 Sticks, towards the end of the day.

Like all 33 Sticks employees, Jason works from a home office. “The Dude” hangs on one wall, painted by an artist clearly inspired by both Van Gogh and The Big Lebowski. An espresso maker sits peacefully nearby. Various inspirational business books and soccer club memorabilia are neatly arranged on a floor-to-ceiling bookshelf opposite the three bay windows that fill the home office with light. We spoke about 33 Sticks at length during my short visit, and the following is edited and excerpted from our conversation and an interview conducted remotely about a month later:

What are some of the challenges you have had to overcome to stay true to the purpose, or mission, of 33 Sticks?

Starting a business is difficult, especially if you are starting it with no money. You will be challenged to compromise your ideals for money…should I take a project just because it pays well even if it doesn’t align with what I want to do? One of our earliest, hardest decisions at 33 Sticks was to refuse to bill by the hour. And we turned down a lot of technical implementation projects because we want to help our clients solve a larger problem, or condition. Sure…we can implement Adobe but we want to help our customers navigate a more strategic journey and we feel that billing by the hour is in opposition to that goal.

Refusing to bill by the hour has been raised to a moral or ethical level at 33 Sticks…why is that? Why does it matter so much?

It’s part of our ideology…we want long term relationships with clients and we want to help them get to that next level. Billing by the hour directly competes with that goal…it rewards consultants for taking a long time to solve problems. Clients want problems solved quickly. By removing the billable hour we remove any incentive to just take our customer’s time. We’ve worked hard to educate the market in this area…and we’ve made a lot of progress…but it’s still a difficult conversation to have and there are companies who simply cannot buy our services without charging by the hour.

It’s also critical for our employees…as a manager I immediately focus on utilization as a key measure if we bill by the hour. By removing the billable hour we’re creating space for consultants to think. One of our best and most difficult decisions has been to remain faithful to our ideals since day one. Our employees can count on the fact that we will always remain true to our principles.

In taking the long term view towards client relationships, 33 Sticks clearly focuses on different measures. How do you measure success if you’re not focused on measures such as utilization?

If someone from an ivy league business school took a look at 33 Sticks they would be quick to point out that we could easily be making twice our current Revenue. There is a lot of money to be made in technical implementation…there is more money to be made by billing more hours or at higher rates. I really believe in the ideas of Yvon Chouinard, founder of Patagonia, who mastered the long term customer relationship. At 33 Sticks, our only measure is whether our decisions are in the best interest of our client’s satisfaction, and the satisfaction of our employees. Yes, we have more work to do to up front to demonstrate the premium value we offer. But once customers start working with 33 Sticks and experiencing the difference, we know we are establishing a long term customer relationship that will pay off. The work almost becomes an excuse to have these deeper, personal relationships. I’m often criticized for encouraging more personal conversations with customers or employees, such as saying it’s OK to treat them like family, to discuss vacations and share in life events. I feel that our focus on the personal, on the human challenges of walking the analytical journey, is a quality that separates 33 Sticks in the marketplace.

By renouncing the billable hour and advocating for remote work, it feels as if 33 Sticks has a social mission to change the way people work. Would you agree?

Absolutely…if we stay on the Patagonia theme and Yvon’s book “Let My People Go Surfing” the fact that Patagonia makes gear or climbing equipment is secondary to providing opportunities for social responsibility and for different experiences. We have a client who works for a 100 year-old publication, not an organization that you would associate with change, and he went to his management and asked to work with his family from Hawaii for a year. That’s the kind of personal change and new opportunity 33 Sticks wants to encourage. Analytics is just a way for us to to effect social change, to help people find different ways of doing work. 33 Sticks works with some very large brands that make billions of dollars. And sure, we can help them make millions more, but that isn’t personally fulfilling and we work with a lot of clients who feel unfulfilled. At 33 Sticks we want to lead people to a better way of doing work and by extension a better life.

The social mission of 33 Sticks is core to its success…and you’ve mentioned that technical ability is a given in the Analytics business. Do you feel that it’s possible to start a business without the technical acumen?

Absolutely…take a look at the story of Charity Water in the book ‘Thirst.’ Scott Harrison started without clue of how to bring clean drinking water to people around the Globe. He had skills as a nightclub promoter, but he had to learn all of his technical skills along the way.

What are some of the challenges that 33 Sticks faces today? And what ambitions do you have for the company?

We spend a lot of time fighting “good enough.” We could make a lot of money just doing work that is good enough for our customers. Hila has been the driving force behind the quality of 33 Sticks’ work. Early on we decided to stick to our highest standards of quality, and Hila champions this ideal with every client engagement.

My ambition for 33 Sticks is to have a larger voice in changing the culture or direction of a business. It’s about scaling our social mission of changing the way people do work to create an environment that is more fulfilling for everyone. And this relates to our passion for working remotely. I should’t dictate where you do your work…I also have no interest in chasing the changes in technology…it’s too hard to keep up with. We have done a lot of work with Adobe over the years, but we don’t want to be perceived only as an Adobe expert. We want to continue to grow our role as personal business advisers for those companies seeking more fulfilling ways of doing business.

Jason Thompson is the co-founder of 33 Sticks, a boutique Analytics consultancy. He is also an incredible chef and aspiring barista…the author can vouch for the latte Jason created with that humble espresso maker.

Impact On Society

Anti-vaccination: what in the data are we talking about?

Anti-vaccination propaganda is testing our immunity against harmful misinformation. The scientific method for evaluating vaccine safety is losing credibility to more socialized minority opinion. And this anti-vaccination dilemma is a petri dish for exploring how we respond to the different ways data is communicated to the public.

Vaccine safety is a hot topic in The New York Times, and two recent articles dig into the data of vaccine safety. The first, “By the Numbers: Vaccines Are Safe” summaries key findings (and data) in a bulleted list such as the following:

Billions of doses of vaccines have been given to Americans in the 30 years of the injury program’s existence. During that time, about 21,000 people filed claims. Of 18,000 claims that have been evaluated so far, roughly two-thirds have been dismissed because the program determined that the evidence showed vaccines were not at fault.

The second, “Vaccine Injury Claims Are Few and Far Between,” is a much longer piece that includes the following chart:

The chart incorporates bullet-style annotations excerpted from the article, such as “After an exhaustive review, federal courts ruled in 2010 that vaccines do not cause autism.” And the chart adheres to good principles of design, such as consistent increments on both axis, that are often overlooked or manipulated by less attentive publications.

The chart does an excellent job of narration…there is a timeline with story points and measurements. Charts are great for reading and research, but difficult or awkward to produce if you’re in an heated debate at the local pub with someone spouting “facts” from questionable sources.

Bullet points tell a different story and the narrative walks us down a numeric path of size and scope. We start with a reminder that “billions” of doses of vaccines have been administered over 30 years and arrive at 18,000 evaluated claims, the majority of which (two-thirds) have been dismissed by the judicial system.

Bullet points work well at the local pub. You can accent each one with a stern poke in the general direction of your less informed drinking pal. The inevitable problem with the bullet point approach is that nuance is often lost. The judicial process involved in dismissing claims that vaccines cause autism is just one example. Your new friend is likely to counter that institutions are all simply puppets of Big Pharma before making reference to a lesser-known doctor who has “proven” the case exhaustively…on YouTube.

I do not know whether the overuse of charts or bullet points have diminished our capacity for dialog. I do suspect that the surplus of quick “facts” and scarcity of attention has eroded patience for the long tale…rational arguments built by weighing different and opposing facts in an inviting narrative best shared with friends over beer or coffee and in the spirit of good company.

Impact On Society

Analytics on a Dime

I started Your Life in Data over two years ago to celebrate the work of the Analytical community, and to explore the ways data are transforming our day-to-day lives. Unfortunately, one such way is the viral spread of misleading information and deceptive analysis.

Over the past few months I’ve thought about the responsibility we share to combat the plague of misinformation. Our primary role as Analysts is to enlighten with objective truth and to communicate with clarity and urgency. We are the scouts and messengers of the modern age.

The Mercury Dime symbolizes these essential qualities with a reminder of our duty as Analysts to protect “liberty of thought.” I collected Mercury Dimes as a kid…something about the design spoke to my 13-year-old self. Today, I find that it speaks to the professional mission I share with millions of other Analysts.

Stay tuned for a new series of posts that explore the use and misuse of data for rationalizing the most important decisions in our lives.

Forward Thinkers

Women in Data Science 2019 Cambridge Conference

1 Comment

On March 4th I had a pleasure to attend the third annual conference for Women in Data Science in Cambridge, MA. After missing it last year (ironically, because my daughter decided to arrive a week before the conference!) and hearing so many great things about it from my colleagues, I was determined to attend it this year and excited by an impressive list of distinguished women invited to present their latest research. The one-hour delay of the start due to a mild New England snow storm only amplified my (and everyone else’s) anticipation.

The conference began with opening remarks by Cathy Chute, the Executive Director of the Institute for Applied Computational Science at Harvard. She reminded us that WiDS started at Stanford in 2015 and is now officially a global movement with events happening all around the globe. The one in Cambridge was made possible by a fruitful partnership between Harvard, MIT, and Microsoft Research New England.

Liz Langdon-Gray followed with updates about the Harvard Data Science Initiative (HDSI), which was about to celebrate a two-year anniversary. She also informed us that a highly anticipated Harvard Data Science Review, a brainchild of my former statistics professor and advisor Xiao-Li Meng, is going to launch later this spring. This inaugural publication of HDSI will be featuring “foundational thinking, research milestones, educational innovations, and major applications” in the field of data science. One of its aims is to innovate in content and presentation style and, knowing Xiao-Li’s unparalleled talent to cleverly combine deep rigor with endless entertainment, I simply cannot wait to check out the first volume of the Review when it comes out!

The first invited speaker of the conference was Cynthia Rudin, an Associate Professor of Computer Science and Electrical and Computer Engineering at Duke. Prof. Rudin started with a discussion of the concept of variable “importance” and how most methods that test for it are, usually, model-specific. However, a variable can be important for one model, but not for another. Therefore, a more interesting question to answer is whether a variable is important for any good model, or for a so-called “Rashomon set” of models.

Prof. Rudin then switched to an example that motivated her inquiry – an article on Machine Bias in ProPublica, which claimed that the proprietary “black-box” algorithm COMPAS that predicts recidivism and is used for sentencing convicts in a number of states, is racially biased. After digging deeper into the details of the ProPublica analysis and trying to fit various models to the data herself, Prof. Rudin came to a conclusion that age and criminal history were by far the most important variables in the COMPAS algorithm, not the race! Even though it is still possible to find model classes that mimic COMPAS and utilize race, this variable’s importance is probably much smaller than what was claimed in by ProPublica. Nevertheless, Prof. Rudin concluded that the “black-box” machine learning (ML) algorithm that decides person’s fate was not an ideal solution as it cannot be independently validated and might be sensitive to data errors. Instead, she advocated for the development of interpretable modeling alternatives.

We then heard from Stefanie Jegelka, an Associate Professor at MIT, who talked about tradeoffs between neural networks (NN) that are wide vs. deep. Even though theory states that an infinitely wide NN with 1-2 layer may represent any reasonable function, deep networks have shown higher accuracy results in recent classification competitions (e.g., ILSVRC). Therefore, she concluded, it was important to understand what relationships NNs could actually represent. Then Prof. Esther Duflo, a prominent economist from MIT, discussed a Double Machine Learning approach that used the power of ML apparatus to answer questions of causal nature, akin to those that, usually, require a randomized clinical trial.

Anne Jackson, a Director of Data Science and Machine Learning at Optum, was the only industry speaker at the conference. She talked about building large-scale applications in the industry settings: from data cleaning, understanding the context, to incorporating the developed model into the business process. “What we really need”, she jokingly said, “is a ‘unicorn’ – a PhD in Math, with MS in Computer Science, and a VP-level understanding of business – to get it right!”. She also cautioned against blindly relying on algorithms and, instead, always translating models into the real world. For example, comparing stakes for false-positive vs. false-negatives, considering model drift, etc. Finally, Anne touched upon the futility of efforts for building and supporting custom software. Moving away from this approach, more and more businesses start to utilize “middleware”, which is a “layer of software that connects client and back-end systems and ‘glues’ programs together”.

Finally, the last, but most certainly not least, invited speaker was Prof. Yael Grushka-Cockayne, a Visiting Professor at HBS, whose research interest revolved around behavioral decision making (among many other things). In her fun and engaging talk, she emphasized the importance of going beyond just a simple point estimation when it comes to prediction. She also reminded us of the effectiveness of crowdsourcing when it comes to forecasting, with such notable examples as The Good Judgement Project, where everyone can provide their opinion on an outcome of certain world event and get rewarded by getting it right, and the Survey or Professional Forecasters, which obtains macroeconomic predictions from a group of private-sector economists and produces quarterly reports with aggregated results. The last part of the talk was devoted to the results of Prof. Grushka-Cockayne’s successful collaboration with Heathrow Airport in applying Big Data/ML approach to improve upon passenger transfer experience, which did not sound like an easy feat! Ironically, the data which proved to be most reliable and was ultimately used in the model came from baggage transition records.

In addition to a strong lineup of featured speakers, the conference offered an excellent poster session, where students and Post Docs demonstrated their ML applications in a wide range of diverse fields, including drug development, earthquake prediction, corruption detection, and many others. All in all, this long awaited Cambridge WiDS conference most certainly exceeded my expectations and I am eagerly looking forward to the next year’s event.