In Conversation with CarGurus’ Sumita Palanisamy: Orchestrating AI Success in Engineering Teams

Introducing A Tale of Two AIs, FeatureByte’s new podcast that explores the future of enterprise applications and the intersection of GenAI, Predictive AI, and software. In this episode, Razi Raziuddin sat down with Sumita Palanisamy, Director of Engineering at CarGurus, for an insightful chat about leading engineering teams in the AI era, and managing the intersection of GenAI and PredAI to create innovative AI applications. 

Sumita is an experienced engineering leader, having spent 15 years across different industries, including insurance, healthcare, ecommerce and tech startups. She leads engineering teams at CarGurus in Boston.

Combining GenAI and PredAI in Enterprise Applications:

Sumita reminded us that the two AIs are completely different beasts. GenAI, PredAI, and, in fact, software, each have unique lifecycles, timelines, and resource demands. Ultimately, your choice of AI should depend upon the business use case and the suitability of each to different parts of the application. 

Whether fine-tuning a foundation model or training a predictive AI model from scratch, it’s important to realize that you may not have the best possible model from day one. Success, Sumita noted, comes from setting crystal-clear expectations, defining acceptable error margins, and iterating consistently post-launch. 

Bridging the Gap Between Engineering and Data Science:

Sumita highlighted the importance of creating a shared understanding between engineering and data science teams. While engineers thrive on well-defined classes and deterministic logic, data scientists often need to navigate unstructured, fuzzy data, and statistical outcomes. Tools and frameworks help bridge this gap, but clear communication and shared best practices are the real game-changers.

Accountability and Management Across Disparate Functions:

AI applications are complex, and managing across functions such as software development, data science, and ML engineering requires discipline. When a production system is dependent on multiple components, how do you make production incidents work? Who takes responsibility for which parts?

Sumita stressed the importance of versioning models, maintaining clear documentation, and establishing a shared playbook of best practices and processes to ensure transparency, traceability, and consistency. Model registries and experimentation tracking should be part of the formal SDLC (software development lifecycle), not just for accountability, but to trace back and see what actually contributed to a certain outcome, whether good or bad. 

Organizing Teams for AI Success: 

At CarGurus, Sumita’s team spans generative AI, predictive AI, and backend engineering. Having software engineers who are trained to become data scientists or play in that sandbox helps a lot because the team is able to manage the codebase all the way through. But the key to success is working closely with data science teams for input, peer reviews, and code reviews. Having an extra pair of eyes helps avoid common mistakes and keeps timelines in check. 

The alternative model to having developers do data science is to embed data scientists into software development teams. Regardless, it is critical that data science is part of the SDLC, otherwise data scientists tend to get isolated results that are sub-par.

Guardrails and Validation To Establish Trust: 

Sumita stressed the importance of considering the ethical and security aspects of AI development to establish trust and avoid negative headlines. This requires working with Legal and Security teams to establish the right guidelines for teams. 

Equally important is to have guardrails and code reviews in place to avoid leaking unnecessary or sensitive information to either the user or AI model. Sumita recommends having a QA agent as a part of your workflow to validate the code and the output being generated for end users. 

Throughout the conversation, one theme stood out loud and clear: successful AI adoption isn’t just about having the right technology—it’s about building the right teams, aligning workflows, and fostering a culture of continuous learning and collaboration.

Tune in to the full episode below: