Alex Leeds, presented “Building Up Local Models of Customers” at a Domino Data Science Popup. Leeds discussed how the Squarespace data science team built models to address a key business challenge as well as utilized a complex organizational structure to accelerate data science work. This Domino Data Science Field Note provides highlights and video clips from his talk. The full video recording is also available for viewing. Also, if you would like additional information on building and managing models within an overall data science practice, then consider Domino’s model-management paper or practical guide for managing data science at scale.
Data Science Practice
In “Building Up Local Models of Customers”, Alex Leeds discussed how the Squarespace data science team approached the business challenge, or question, “where, and in what cities, should we be doing targeted localized marketing"? Within a few weeks, the data science team addressed the business question by building a model that predicts success and enabled the business stakeholders to determine which cities to focus on. The data science team was able to build the predictive model (and many other models) within a few weeks because they already had an ecosystem in place. An ecosystem that included, but wasn’t limited to, the data science practice.
Leeds indicated that a “data science practice depends on a great deal of other steps” that include data collection, qualitative research, and market testing (i.e., over 30 experiments a year) which “feeds the kinds of analysis that we do in our models as well”. Leeds also noted that data scientists and data engineers working “side-by-side” is a “huge advantage for the company” because it enables data scientists to “get at the data very quickly” and “have confidence in the information we were drawing on, just to even begin”. In the talk’s conclusion, Leeds also noted that there were “a lot of models that went into the analysis”, how “data science models build on each other”, “model diversity forces skill set diversity”, “and interacts with organizational process”.
Approaching the Business Question, or Challenge
In the talk, Leeds outlined that the approach the data science team took to address the business question. The approach included identifying the cities where Squarespace was doing well, “gather[ing] location-specific data”, “determin[ing] drivers of performance” and “us[ing] these drivers to inform business strategy”. This approach enabled the team to discover that some of the factors that “mattered” or predicted success at the city level included the percentage of people within 30-44 years old. The potential contextual implication is that people within this age range are building businesses and use websites for their businesses. This was an interesting discovery as the stakeholder’s hypothesis was that a younger demographic would have been the predictor of success. Leeds also noted that there were many additional indicators that predicted success but that just two additional indicators included percentage of people with graduate degrees and number of food carts.
While the predictive indicators are interesting, yet, how did Squarespace use these results to impact business strategy? Leeds noted these results helped Squarespace identify which locations to focus on.
Implications for Doing Data Science In Practice
Leeds concluded his talk by using this project as a vehicle to discuss “doing data science in practice”. Leeds noted that while the team built a model to predict success, there were “a lot of models that went into this analysis.”
He indicated that “models build on each other” and enabled the team to “do this very rapid analysis” and provide the organization with specific recommendations. Leeds also advocated that there is “some intriguing diversity” within the model set. He also noted that
“these sources of data and sources of models are going to create this interesting organizational process. You need to have data scientists that are close enough to certain branches of your organization to build informed models. And even in this really clean, really straightforward, very common question, how are we doing in different cities? Where should we be focusing next?….We've actually described a system that has a lot of complexity in terms of the organizational structure as well as the practice that you need.”
In “Building Up Local Models of Customers”, Alex Leeds provided insights into how the Squarespace data science team addressed a key business challenge by building models and an organizational structure that accelerated data science work. While this Domino Data Science Field Note provided highlights from the talk, the full video is available for viewing. Also, if you would like additional information on building and managing models within an overall data science practice, then consider Domino’s model management paper or practical guide for managing data science at scale.
Domino Data Science Field Notes provide highlights of data science research, trends, techniques, and more, that support data scientists and data science leaders accelerate their work or careers. If you are interested in your data science work being covered in this blog series, please send us an email at writeforus(at)dominodatalab(dot)com.