Driving Customer Value and Efficiency by Transforming Model Development and Deployment

Data Science at Moody’s Analytics

Moody’s Analytics supplies expertise and tools—including data, models, software, and professional services—that help companies grow efficiently and manage financial risk. The firm supports thousands of organizations across a range of industries and geographies as they navigate increasingly complex global markets. Moody’s Analytics is a subsidiary of Moody’s Corporation, which reported $4.2 billion in revenues in 2017 and operates in 41 countries.

A pioneer in financial modeling, Moody’s Analytics has established a competitive advantage by creating analytics based on unique financial data sets and applying them to solve clients’ business challenges. The firm has developed a large portfolio of models covering everything from small business to large bank credits, and deploys its models in software delivered on-premise, in the cloud, or as software-as-a-service (SaaS).

By centralizing some of its data science projects on Domino, Moody’s Analytics has dramatically increased the efficiency of model development, and expanded its ability to build collaborative models with clients and partners. To make this transformation, Moody’s Analytics needed to accelerate the pace of model development and the number of iterations they could make on models under development. They’ve accomplished this with Domino, reducing end-to-end model development cycles. For example, in its first experience using Domino, Moody’s Analytics reduced an estimated nine-month project to four months. The firm has also increased its model monitoring capability by four times using Domino.

Challenge

Moody’s Analytics has been a long-time leader in financial modeling, and over time its models have been widely adopted and have become vital to many firms. To stay at the forefront of an industry that runs on models, Moody’s Analytics saw an opportunity to customize its offerings to satisfy particular clients’ needs, profiles, and objectives. The company had already been successful with a services-based model whereby Moody’s Analytics consultants build custom models for clients in the field, but this business model is expensive, both for Moody’s Analytics and its customers, especially at a large scale. Moody’s Analytics sought a more cost-effective approach to development and delivery of these customized models, in order to serve its customers more effectively.

Key challenges for Moody’s Analytics in transitioning their customized model delivery services offering into a product included:

• Access to tools and technology: Data scientists struggled to get access to the infrastructure supporting the dynamic computing capacity they needed to be able to build and test models.
• Model development and delivery friction: With a development and deployment workflow that was linear rather than cyclical, there were long delays in getting models into production and improving them over time. “Creating a controllable, auditable, repeatable, and transparent process took too long,” said Jacob Grotta, General Manager of Banking Operating Unit at Moody’s Analytics. “The cost of improving or replacing a model was too high.” In a competitive industry, the company needed to get its risk models into customers’ hands sooner to increase their return on investment.
• Silos of knowledge: The company’s globally distributed teams couldn’t easily collaborate or repurpose others’ work because different users and teams relied on different tools such as R and SAS. They also risked losing important institutional knowledge when a key team member departed. To continue building world-class models, they needed a single platform where teams could collaborate like the company’s software developers have for many years.

“Working with Domino gives us the ability to fully leverage our core strengths of working with financial data, building analytics, and applying those to our customers’ businesses ”

Jacob Grotta

General Manager of Banking Operating Unit at Moody’s Analytics

Solution

Moody’s Analytics first considered building a solution in-house that would expedite model development and delivery. However, advanced capabilities within the Domino data science platform would be able to accomplish the company’s objectives while allowing internal teams to remain focused on modeling – their core capability – instead of systems development. “Working with Domino gives us the ability to fully leverage our core strengths of working with financial data, building analytics, and applying those to our customers’ businesses,” Grotta said.

Several Domino capabilities made its data science platform the most attractive option:

• Full lifecycle support: Domino doesn’t just help data scientists build models, it facilitates the end-to-end workflow – from ideation to discovery of existing work, model sharing, deployment to production (via either APIs supporting external models or human consumption), quality assurance, monitoring, and documentation. This end-to-end process enables rapid model delivery and iteration which translates to faster innovation and competitive differentiation.
• Openness: The company’s data scientists and analysts around the world use a range of tools and languages, including RStudio, Jupyter, Python, Apache Spark, and SAS. Domino allows all users to collaborate while using the tools they prefer in their native development interface.
• Reproducibility: Domino automatically tracks the lineage of every model in a centralized location, so the company can easily reconstruct everything including package version, code, data, parameters, results, and comments in each experimental iteration of model development. This is critical for regulatory compliance.
• Collaboration and knowledge management: A centralized platform with automated tracking of data science work means teams onboard newcomers easily, build off of past work, and preserve knowledge in case of turnover. It also gives managers the transparency needed to make rapid monitoring decisions. This particular capability helped Moody’s Analytics achieve a key business goal of faster model iteration by eliminating duplicate work and enabling each iteration to build off the previous with full transparency.
• Infrastructure automation: Domino acts as an orchestration layer on top of Amazon Web Services (AWS), providing flexible, elastic access to computer resources in the cloud. Users aren’t tethered to their workstations, and with one click, they access their exact tool of choice on the server size needed. What used to take days of installs and configurations is now available in seconds.

Moody’s Analytics has deployed Domino to build new models and to monitor, manage, and enhance existing models once they’ve gone into production. Domino runs on AWS, allowing users access to the enterprise-scale platform without concerns about their security or location.

The Domino Effect

Moody’s Analytics serves its customers better today because the company can deliver models that perform well and are custom-built for each customer’s business – and can tweak them rapidly to ensure they continue to deliver more value after each iteration. The end-to-end process from model development to deployment into production has been reduced by more than half, from nine months to four. They’ve improved ROI by streamlining model development and deployment. The centralized platform and automated tracking has also enhanced transparency, preserved institutional knowledge, and improved compliance.

“With Moody’s Analytics know-how and workflow, coupled with Domino and AWS infrastructure, we have been able to accelerate model development, which means information gets into the hands of our clients who need it faster,” Grotta said. “Our customers are excited because their needs are going to be answered in new ways that would previously have been impossible.”

Two examples demonstrate the impact Moody’s Analytics has achieved with Domino:

• The Moody’s Analytics Financial Statement Quality Checker, available as part of its MARQ™ Portal, uses millions of records to flag interesting information for customers, such as whether a company’s assets compared to its liabilities have changed too quickly relative to peers or the previous year’s statement. Using Domino for model deployment, data scientists were able to develop an API and share it with customers for beta testing within a few days. They used feedback to make adjustments and redeploy the model almost instantly, eventually embedding it into a product release. “Rather than taking a year, the process took a couple of months, and the cost of deployment was much lower,” Grotta said. That’s a 6X performance acceleration.
• Moody’s Analytics suite of 40 RiskCalc™ models is used to estimate companies’ probability of defaulting on debt across the globe. Moody’s Analytics customers frequently are interested in questions such as, “What if the model was developed on data specific to my regional footprint?” or “What if the model used a particular financial ratio my firm pays attention to?” Moody’s has developed templates leveraging Domino and AWS that allow it answer these questions – and to deploy alternate models to customers for testing and use in short order. It can also interact with its customers through this process, giving customers the level of transparency they need. “We are now able to answer customer questions and adapt our industry standard models to customer requirements in a very short amount of time,” Grotta said. “The efficiency and repeatability makes this far more affordable for our customers.”

With Domino, Moody’s Analytics can efficiently deliver customized models for risk and other analytics that help run large-scale enterprises, and cost-effectively deploy them according to customer preferences on-premise, in the cloud, or as SaaS.

“We’ve been able to standardize the data, the know-how, and the ways of collaborating amongst ourselves and with our customers so that they can see the work we're doing as we do it,” Grotta said. “Domino accelerates our speed to delivery, providing a much faster and better return on our modeling investment.”