Subject archive for "engineering," page 2

Many thanks to AWP Pearson for the permission to excerpt "Manual Feature Engineering: Manipulating Data for Fun and Profit" from the book, Machine Learning with Python for Everyone by Mark E. Fenner.

Even the most sophisticated data science organizations struggle to keep track of their data science projects. Data science leaders want to know, at any given moment, not just how many data science projects are in flight but what the latest updates and roadblocks are when it comes to model development and what projects need their immediate attention.

Our last release, Domino 3.3 saw the addition of two major capabilities: Datasets and Experiment Manager. “Datasets”, a high-performance, revisioned data store offers data scientists the flexibility they need to make use of large data resources when developing models. And “Experiment Manager” acts as a data scientist’s “modern lab notebook” for tracking, organizing, and finding everything tested over the course of their research.

Our mission at Domino is to enable organizations to put models at the heart of their business. Models are so different from software — e.g., they require much more data during development, they involve a more experimental research process, and they behave non-deterministically — that organizations need new products and processes to enable data science teams to develop, deploy and manage them at scale.

Eugene Mandel, Head of Product at Superconductive Health, recently dropped by Domino HQ to candidly discuss cross-team collaboration within data science. Mandel’s previous leadership roles within data engineering, product, and data science teams at multiple companies provides him with a unique perspective when identifying and addressing potential tension points.

This blog post provides highlights and a full written transcript from the panel, “Data Science Versus Engineering: Does It Really Have To Be This Way?” with Amy Heineike, Paco Nathan, and Pete Warden at Domino HQ. Topics discussed include the current state of collaboration around building and deploying models, tension points that potentially arise, as well as practical advice on how to address these tension points.