As an undergraduate in India in the field of Electronics and Communication Engineering, Suju Rajan became fascinated with the field of Machine Learning. In 2001 she came Stateside to pursue a PhD in Machine Learning at the University of Texas at Austin and since then, has built a career and expertise in Data Mining and Machine Learning. She joined Criteo in May of 2016 to grow our research team and is now in charge of leading Criteo AI Lab. This week, we sit down with Suju and talk all things Machine Learning, and learn what sets Criteo apart!
Let’s start from the beginning, where are you from?
I got my undergraduate degree in India in the field of Electronics & Communication Engineering. My research project for my final semester was on building neural networks to recognize handwritten digits from corrupted/noisy images. The project was fascinating and that led me to apply for a PhD at UT, Austin under Professor Joydeep Ghosh.
While at UT, I worked on a bunch of interesting ML problems in collaboration with some folks at the Center of Space Research, mostly towards learning to classify the pixels from hyper-spectral images into different classes of vegetation/land cover types. I also had an excellent summer internship at my final year at Yahoo Labs. By then I had decided that my career lay in applying ML to real-world problems.
How did you start your career? What did you do before you started at Criteo and what brought you to Criteo?
I started my career as a member of the Data Mining & Research group at Yahoo. The role of the group was to provide Machine Learning models to other Yahoo products. I learned a lot about many different applications of machine learning there from predicting conversion leads to producing clusters of news articles in real time.
From there on, I eventually got to direct a group of research scientists to produce machine learning solutions for user personalization on a number of Yahoo products news, apps & videos. I also had an excellent manager & several incredible mentors from whom I learned how to build and make a research team be successful. Around this time, Criteo provided me the opportunity to build out a world-class Research team and help establish its impact internally & externally and of course, I had to take it.
How has your role changed since you started?
I distinctly recall my first conference attendance at Criteo: KDD 2016 in San Francisco. I was a bit disappointed by the number of folks that came by our booth who did not have a clue as to what we did as a company. It was critical for us to establish Criteo Research as a ML industrial lab of significance.
With the help of a lot of incredible Criteo folks, we have turned the tide around and now people stop by our booth with the intention of applying for a job or seeking some sort of a collaboration. We need to keep up our pace of external impact to maintain this position.
Similarly, internally within Criteo, very few people even had an idea on how to leverage the Research team. It wasn’t clear how the team worked or how to approach us for collaborations. We’ve made a lot of progress on this front as people are more aware of the charter of the group. We are engaged with most of the major Product initiatives underway at R&D.
How has Criteo helped foster your career to where you are today? How have you taken advantage of opportunities Criteo presented?
I was able to help create a Research team of significance only with the unwavering support of my manager and a whole slew of supporting teams. We had created a number of unique programs from Faculty Research Awards and internal ML Bootcamp classes to get folks familiar with the Research team.
All of our efforts have had enthusiastic participation and encouragement. Personally, I have had almost complete freedom to decide and shape this Research team and to represent it externally. I have embraced these opportunities and learned a tremendous amount in the progress.
What’s the biggest difference you see at Criteo from other companies you’ve worked at? What has been surprising and enjoyable so far?
Machine Learning is truly in the DNA of the company. It’s not just the purview of the Research team. There are plenty of excellent machine learning engineers spread throughout R&D and we have a Product team with an equally excellent grasp of the field, so it’s incredibly easy to have a detailed conversation with folks. It also ups the ante for the Research team and makes it possible for us to focus on riskier, harder problems.
What’s the goal of your position? What’s the hardest thing about it?
Research by nature is about long-term bets and risk-taking.
Let’s say our current production systems are running Model A and we as the Research team think it is better to run Model B. This hypothesis is backed by a lot of offline experimentation, but the reality is we’ll eventually have to run it in production, at scale, to learn if it works.
The question is now: how do we trade off the effort that is required to put Model B into production with other priorities? How do we justify other work that may get put into the backlog because of this hypothesis? The path forward is to reduce the cost of such changes and experiments. How do we do this without missing out on other opportunities is the challenge we face.
What projects/achievements to date are you most proud of?
We grew the team from eight to more than twenty-five people within one and a half years, without compromising on the quality of hires.
Externally, the Research team has really stepped-up our presence from publishing papers in a bunch of well-acclaimed conferences, including winning a finalist award at the AdKDD conference in 2017, to establishing academic contacts with a number of universities through collaborations and internships. We are no-longer an unknown in the ML academic circuit.
Internally, our Machine Learning Bootcamp project has really taken off. We are now in its fifth edition and have provided in-depth practical training to over 20 engineers. It is great to see the graduates of this program then subsequently apply this knowledge back to projects in their parent teams.
Let’s talk about Machine Learning and Artificial Intelligence. How is Criteo leading the way?
Criteo pretty much leads the field of advertising technology because we are laser-focused on improving the performance for our clients. Online advertising—the main economic model behind the Internet—is evolving. Our ability to face these changes must rely on strong fundamental research dedicated to the creation of new principles, models and algorithms.
So, on Wednesday, June 27th, we announced the launch of the Criteo AI Lab, a center of excellence dedicated to the creation, experimentation and at-scale deployment of machine learning technology. Backed by an investment of €20M over three years, the lab will be involved in research related to building deep models that are interpretable, transparent and user-centric.
We don’t shy away from optimizing the best possible objective function for our clients & constantly try to get to this state. We have tons of super experienced folks who are pushing the way forward in figuring out how best to use the data we collect on our advertiser sites, our users and our publishers. Very few companies have this 360-degree view of the impact of advertising and we are well on our way to leveraging this.
Hobbies outside of work?
I have too much fun with my two toddlers to leave time for much else outside work these days. Ask me again in a couple of years!
Lastly, any advice for newcomers or people who are looking to build a similar career trajectory?
Ask for what you want. Don’t hold yourself back from signing up for tasks that are clearly new territory – you will learn if you want it badly enough.
Thank you so much, Suju!