Code + Culture: Sidu Ponnappa & Ajey Gore on Scaling Engineering Teams and Product Development

SHOW NOTES

• Product to process: Redefining success metrics [02:54] 
• Feature fallacy: Rethinking the role of product engineering [06:03]
• Navigating the black box: Why decision-makers need to be in sync [11:43] 
• The people paradigm: The human factor in hiring strategies [16:31] 
• Why you should treat employees as customers [22:58]
• Even potential employees can be your brand ambassadors [30:43]
• On candidate segmentation: One size doesn’t fit all [34:14]
• On metrics and scaling data engineering teams [40:53]
• Optimizing collaboration: A developer’s perspective [51:26]

TRANSCRIPT

Ajey Gore: Let me give you a little bit of context. I always say that it takes a village to build something. Our latest journey was Gojek, when we did something, Sidu was a very important pillar. But before that, let’s go back 16 years, Sidu started a startup called Activ Mobs, which was Twitter [X], but with SMS. And they created communities in Bangalore, they ran it for two to three years, and they got a lot of traction, it was pretty good. But, Sidu has this fortunate or unfortunate thing: He’s always ahead of time. So that did not pan out. 

Then Sidu worked with ThoughtWorks with me. We were there for four years. But there was always an urge to surge, and Sidu actually went out again and created a company called C42 [Engineering], which was a consulting company, which ended up creating an EdTech platform for learning Ruby and other technology platforms. Then, they said, enough of consulting, let’s do a product. So, like you say, “You should eat your own dog food,” they started a matrimony product, and got one of the co-founders married! Which is good, because they were trying to figure out the product and the co-founder found somebody on their own product and they got married, which is perfectly fine. Niranjan [Paranjape] is still there. 

And then we got into working with Sequoia around early 2015. And late 2015 both our companies got acquired by Gojek, through Sequoia. And then, Gojek went on a crazy path, we grew around 5% to 10% week-on-week. Which means that if you were doing 10,000 orders in May 2015, you are doing a half a million orders in December 2015. You are doing 1.5 million orders in December 2016. And you are doing 4.5 million orders in December 2019. So we grew [like] that, with payments growing to 7.5 million orders. And while all that stuff was going on, and a lot of people see me as trying to do that, Sidu played a very important role in building our org and scaling our recruitment and making sure many more things… So Sidu headed HR, recruitment, he was dealing with India, as in he was MD India [of Gojek Engineering Centre] and also did data engineering. So he did many things in those five years. That’s a summary of what you have been doing. So we want to talk [about] those days where he has been three or four times, and done things which are around org, hustle, recruitment, the problems you are already facing, or you all will face in say the next two, three, four quarters, if you guys find PMF (product-market fit) and all this stuff. Actually, in the early days of PMF, Sidu, we did a lot of stuff. How do you think we should think about product development in the early days?

Product to process: Redefining success metrics [02:54]

Sidu Ponnappa: This is a very interesting thing because over time, and I’ve gone through this cycle a few times, right? It took me a while to realize that pre-PMF, you aren’t really building a product. It’s dangerous to think of what… A lot of what we do is actually predicated on how we think about it because nothing that we do is physical. If you’re building a car or something, it’s very easy to see what you’re doing, “See, I built this part. I built this part.” When you’re building a company, and especially something that depends heavily on software, most of what you do, you can’t see. And it’s very important to realize that how you imagine it and how you think about the mental models, dramatically changes how you execute. 

And the shift was when I realized that pre-PMF, what am I really building? I can’t possibly be building a product, because I don’t know the product. I can’t possibly be building something that I don’t understand. So over time, what happened is my mental model shifted to thinking about it as, “What I am building is an experimentation and search engine. There’s a solution space in front of me, within which there is PMF. And I have a certain budget. I have a certain amount of time. And I have access to certain other resources, but fundamentally it’s time and money really. What I’m building is a machine that can search the solution space most efficiently to find PMF. Once I find PMF, then I pivot to actually building the problem. They’re not the same thing at all.”

So, let’s talk about this through, for example, metrics. It’s very easy to anchor on the metrics of your domain and your thesis. If you’re building EdTech, whatever, you will start thinking about student minutes on the platform etc., pre-PMF. But the reality is those are all second order. Your core metrics are how much does it cost me to run an experiment? How long does each experiment take? Like, I literally need to have a project management dashboard that says, “Experiments. Those are my units. Cost per experiment, time per experiment.” 

And then, probably most importantly, does each completed experiment emit information, which allows me to make a decision. If an experiment runs and I don’t make a decision after that, it’s easy to think about it as, “Did the experiment succeed or fail?” You want the experiment to succeed, but that’s still the wrong metric model. You want the experiment to emit information, which allows you to make a concrete decision. So the third metric is, “Did I make a concrete decision after this experiment? Yes, no.” It’s a completely different way [in which] I started looking at it. And, that really changed how I executed day-to-day. Literally, what do I do at the start of the day? What are my metrics? How do I track progress, week to week, month to month, everything changed.

Feature fallacy: Rethinking the role of product engineering [06:03]

Ajey: Tell us about one of the things which we did at Gojek in terms of experiments, and we went through this thing which says, “We are not a feature factory. And we should not treat”… So one of the things we realized, a lot of time product engineering is treated as a feature factory. You build features and [more] features. We just keep expecting that from product  engineering teams. And Sidu says, why [we should] not build features, and I want to ask you, why [we should] not build features? What is it that actually keeps you away from treating the product engineering [teams] as a feature factory?

Sidu: So, this is my favorite rant. Most conversations with founders or peers in the ecosystem involve a conversation about features they want to build.

And fundamentally there’s nothing wrong with it. But I like to take an extreme stance on it, which is, please do not build features, because this falls into the same class of problem. And it’s interesting, what is a feature? I have this exercise that I like to run through with a lot of words that we use very casually. Where I’m like, “Can you sit down and write a one-pager that explains the definition of this word?” And then, will that definition match? To a reasonable degree with most other people. And there’s an amazing percentage of the vocabulary that we use that fails this test. Product, by the way, is one of them. What is a product? Sit down, write a one-pager, get someone else to write a one pager. It will not match. Feature is an extremely common example of this as well. 

So what ends up happening is the moment you start thinking of yourself as building features, it affects everything about your operational flow. It dramatically changes how you execute. And this is weird because if you can’t define what a feature is, how is this something that shapes your execution? So the way I like to reify and phrase it is to say that, “Look, [a] feature is not a unit of execution. A feature is a unit of distribution. A feature is something that we package and sell to our customers. A customer buys a feature. We don’t build a feature. You can’t build a feature.” What you can do, is to deliver certain value props, which usually, though not always, can be quantified into metrics. We can say that, “Look, metric X is here. We want to move it to plus, so many percent. And we’re going to run these following experiments, which will attempt to move the number, the experiment that succeeds.” Awesome. We stick with it. We discard the rest. 

And then when you have a basket of value props, which have palpably improved for the customer, you pick those, package them into units and sell them. And those packaged units are called features, because I believe the customer perceives and will be more amenable to paying for metric A, metric B and metric Z, packaged together into a unit.

But the moment you start thinking of yourself as I’m building a feature, what actually happens is think about it. Think about them. What happens in your mind when you say, “I want to build a feature.” You imagine a customer, picking up their phone, opening your app. I’m assuming you have an app, but whatever the interaction is, and then you imagine them going click click, click, click, click. Wow. Value prop. Customer [is] thrilled and pays you money. This is what we imagine. We imagine the customer interacting with the product. 

Now what’s happened is you are excited now because you’re vested. You’re excited and now biased in the interaction. You are now already committed to making that interaction work. And unfortunately, a workflow on an app is not an experiment, nor is it a unit of execution. What your target is the value prop at the end. So the moment you start thinking about it in terms of, “Oh! Customers will do this. [Customer will] get excited, blah, blah.” You’ve already biased yourself. You’ve crippled your ability to design and run ‘n’ experiments, attempting to deliver that value prop. Maybe there is no workflow needed at all, right? Maybe there is a manual process. Maybe there is something else, which is a side effect somewhere else, which can deliver that value prop. Maybe there’s a cheaper way to do it. Maybe there’s a better way to do it. You’ve automatically biased yourself. So the moment you think [about] features, what you’re actually doing is you’re running a fantasy in your mind of a customer interacting with your product in a very specific way, rather than staying focused on ‘I want to deliver value prop X to the customer, what are the 10 different experiments I need to run to get that value prop in the hand?’ 

So the reason I argue against features is not because there’s anything fundamentally wrong with it, but because the degree of bias it creates is ridiculous. So instead of focusing on delivering the value prop, you start anchoring on proving the success of the workflow. You will see this repeatedly if you start digging into the project plan. Your project plan starts skewing towards running experiments to prove that workflow will succeed. And the worst sort of slippery slope outcome of this is what Ajey just described as a ‘feature factory’. Where we used to start the year with a whiteboard with blocks of stickies. Here are the 200 features we will build this year. Because the assumptions have already gone in that if these features are built, these value props will get developed.

Navigating the black box: Why decision-makers need to be in sync [11:43]

Ajey: So, let’s flip this the other way round saying, “If you do not want to build a feature factory…” A lot of product engineering people actually face this. Yesterday we were talking, Shailendra (Singh) and I, and there was one quote which says, “You’ll always be on fire if you’ve found the PMF and the product engineering will always be slow.” You’ll always feel that your engineers are slow, they don’t deliver a lot. How do you want to advise on that? How should CTOs and CPOs, head of product, head of engineering, look at this? What way should they interact with their founders? Or what way founders should interact with the product engine? Like, we have dealt with a lot, at Gojek. What did you do over there?

Sidu: Look, this is a difficult one in many ways. 

Ajey: And the…Sorry to interrupt you, the reason I’m asking this is that time is of essence in the early days and we shouldn’t be spending a lot of time debating [this]. Also, we shouldn’t be spending a lot of time debating on what we should build. We should build appropriate things. So that’s why I’m asking.

Sidu: Yeah. So this is a difficult one for me to generalize, I want to caveat this up front, because a lot of it is down to the chemistry of the decision-makers. It’s not a rational thing. I will give you post-fact, like after this point, I will give you a rational framework, that I believe in, but the truth is, yeah that’s going to be something that you use to support you to a degree. But the reality is that the core of this is the chemistry between those two people. Typically your CEO and your CPTO (chief product and technology officer). I’m not really a fan of splitting CTO and CPO, at all, doesn’t make sense to me. But those two personas, the chemistry between them is of paramount importance. I’ve seen an entire spectrum of work, like, one dominant, one always this thing, both peers, it just is a snowflake. 

So having caveated that, it’s very important to understand that your execution, in this game, very quickly becomes a black box. Now running an analogy from manufacturing, which is a dangerous analogy again, caveat up front. In let’s say car manufacturing, your R&D is a distinct upfront step. You design the product, iterate on it, whatever, and then once it is done, designed, then you go to production, and then you scale up production. I want to caveat that for us, those two phases are combined, perpetually. R&D and productionizing it are essentially the same thing for us. 

Having made that caveat, when you look at a car manufacturing system, if you say you want to improve the car, some attribute of it, you don’t work on the car, that’s meaningless. You work on the production system. You’re like which assembly line requires what different tools and machinery? Where do I need more manpower? Where do I have inventory piling up? Are there bottlenecks? Do I have quality control issues? Those quality control issues are not…you don’t quality check the car at the end, right? Or you do, but that’s not all you do. You’re root-causing it down to the individual components, which are failing. And then you’re attacking quality control at a component level on the assembly line. This mental model is for everyone, it’s a fairly straightforward manufacturing thing, right? 

Yet for us, what actually happens is in the early days, it’s two, three, four, five of us hanging out together, building stuff everyone can see. Everything is cool. Everyone knows… you scale up a little bit. Boom. The factory is a black box. All you see is the defects in the car, your end-product in the hands of the customer failing. You have no idea where those defects originate. You just make assumptions. If I ask you to scale up manufacturing, literally, the only thing you will be able to call out for sure is ‘I need more manpower’. You cannot tell me which assembly lines are blocking. You can’t tell me where you have inventory, like literally, all the standard scaling levels in manufacturing, which do apply to us, are invisible. It’s a black box. So this is why chemistry matters, by the way, because instrumenting our factories is very hard because they are not physical. It’s an observability problem. And consequently, what ends up happening then is it is down to the subjective opinion of those two leaders as to what the problem is and how to solve it. And without instrumentation, just statistically speaking, you’re going to be wrong most of the time. And inevitably you’re going to come down to one of two answers. We need more money. We need more people.

The people paradigm: The human factor in hiring strategies [16:31]

Ajey: So let’s talk about people. In the early days, in C42, you hired a lot of people. Then Gojek, Gojek was tough for us because we didn’t operate in India. And we had to hire a substantial number of people in India because scope was too much. Not because we wanted to go over quantity over quality. So tell us, what did you do? You did a lot of stuff. Tell us about the recruitment journey, because that was something pretty interesting. People still call us and tell us that Gojek is a strong and leading brand in India. So what did you do? Tell us that journey.

Sidu: So, I’m going to adjust this to be relevant to you folks. Because a lot of that thing was [about] starting to run machinery at scale. Where you need to consistently roll out at the 99th percentile, 20 offers a month. And while that is not like service industry scale hiring, at a product industry scale, that’s non trivial, right? Like you’re onboarding in excess of a hundred people. And for us in India, there were no operations at all. You’re just pure hiring tech. So I don’t think that’s super relevant because that is a lot of operations research, honestly. And it’s just like straight up numbers on pipelines and optimizations.

In the early days, what really mattered, and I think this is true for all of you as well, is that it’s very crucial to look at candidates and understand their motivations. And the most important thing to understand is they are never going to join your company. And they’re never going to join your vision. They’re going to join you. It isn’t the vision that moves them. It is your articulation of that vision. Literally anybody else articulating that vision will not change their mind. You are the number one selling point. And I don’t mean this in a Shah Rukh Khan-movie star sense of you being the selling point where you’re a brand ambassador. I mean, specifically it is the opportunity to work alongside you. To get your time and attention, over the next year or two as they work in your company, that is the number one value prop for most candidates. Now, you have to design your hiring strategy and your hiring ops around this. 

It’s easy to say, but as you start ramping up, you have to be able to structure your hiring funnels, like surfacing the information, so that you can take coffee conversations with candidates who cross a certain threshold, and I do not mean cross a certain threshold in your funnel. Who crosses a certain threshold of a high quality candidate because you mean you have, may have zero hope of affirming that candidate until your next round, right? 

You may not have an open position that’s relevant for that person. That person may be too skilled for what you have, or they may be, they may have adjacencies, right? You may be looking for a VP Engineering where most of your focus is on the backend. You may need someone who is kickass at the frontend. Too many founder’s I see who are like, “Oh! You are not relevant to me today.” It’s so important to build that network. The metric [against which] you want to track your own performance is referrals: How many candidates I have spoken to, who are referring more candidates to me. This is the metric you live and die on. It’s an operations problem. You are going to find yourself having two coffee conversations per day, on top of everything else that you are doing because if you don’t make that investment, you have to understand it’s a network, it’s compounding. If you don’t make that investment now it will never compound. So you have to do those things, and track your numbers in how many referrals am I getting from people who we didn’t hire because obviously in a larger company referrals are mostly internal, right? It’s like employees referring [other] people. But the same dynamic applies here throughout you sell. Remember, what you are selling is not the company, not the vision, you are selling working with you, in that company, with that vision. 

And you are selling this to everyone who crosses that threshold whether you can afford them, whether you intend to hire them or not. And you track your performance on that by saying, “Boss, how much inbound am I getting off this?” 

And the reason I’m saying this over and over again is so many people just back away from this, because on top of everything else, you’ve got to do it. It’s literally like you’re bucketing like two hours of sales, at least three days a week. But if you don’t do it now, it will not compound. And by the time it comes to the place, we actually need to continuously hire a couple of people a month. Your funnel won’t be there. So this, I think, is the approach that I would extract. Like, we did this, but we did this with key technologists. And when we got the chance to pull people over from Indonesia, when they were visiting, we would do it with them as well. But we basically built a recruiting strategy around the idea that here is someone you would love to work with. You should join because you get to work with this person.

The typical approach that we see from large company hiring is the company brand makes the hiring manager. You go join an Apple or a Google, I mean the hiring manager has the respect, yeah, they would have done amazing stuff, but the lion’s share of the attention at a brand level, comes because of the company’s brand. It’s completely the inverse here. The lion’s share of the attention to the company’s brand comes because the people are amazing. 

At some point of scale, it will flip over. But for a very, very long time, it’s the other way around. So you structure your entire branding around the people. And to begin with, that’s you. My point here being that as your organization scales up, you can diversify your branding strategy to start building the brand around your key people who are amazing and market the opportunity to work with them as a key factor. 

Why you should treat employees as customers [22:58]

Ajey: One thing, which you did and which was talked a lot about, is you treated every organization problem as a product. Tell us about that. How did you treat…So I’m going to split it into two big organization problems and how did you treat them as a product? First, how did you treat recruitment as a product? What did you do over there? And then, how do you treat org scaling as a product? We’ll come to that one later, but tell us, what is this product thought process, which you applied to almost every organization problem. 

Sidu: So recruiting is an interesting one because I ended up talking to folks a lot over that because a lot of my personal public branding is along those axes. Basically, I index my public brand along stuff like recruiting and engineering because it fits. And so a lot of inbound conversations around that. And when I talk to folks about recruiting, there’s a lot of blunt instrument stuff going on there. It’s like, “Yeah, I want to hire engineers.” Okay, “Tell me what do the engineers you want to hire, like? What do they like?” They’re like,  “What do you mean?” I’m like, “Okay, you’re building a product, right? So, what’s your target segment [of customers]?” They’re like, “Okay, it’s segment A, B, C. Age range this, blah, blah..” I’m like, “Okay, tell me a little bit about that segment [of customers]. What do these people like?” They’d be like, “Yeah, they typically drive these types of cars, work in these types of jobs, have these kinds of…” I’m like, “Okay! You want to hire engineers, so why don’t you have the same level of detail on them? It’s the same logic, right? It’s an acquisition problem. 

So, the point I think that Ajey and I were discussing here is, there are very powerful mental models that we routinely use day in and day out for our product execution, without even thinking about it. Ten years ago, that stuff was rare, the lean startup stuff, segmentation, but today this is very normal. All of us know it, you go to any meetup, talk to anyone, the language is normal, but somehow it starts and stops with what we call the…which is why I asked the question, what is the product? What is this thing called the product? And we generally tend to think that the product is the app or whatever equivalent website, whatever equivalent API, whatever it is that we’re making and selling is the product. But that’s not true. It’s all interconnected. And if you use this lens, you can literally look at any problem that you need to solve and use this mental model and say, “Look, acquisition, engagement, retention, segmentation, behavior, value props, points at which engagement happens, all that stuff that we do on the app applies everywhere.” 

So with recruiting, when you start thinking about it this way, what is the product model that you would apply to this? Product model is, I have customers in the market who have options. We call these customers candidates and that messes with our judgment. Start calling them customers. 

We get this dramatic shift in our mental models by calling them customers, because when we say candidate, and I know that, this is going to sound a little harsh, but it’s the truth. At some level we think because we are paying them that this is different, but the truth is we’re running for-profit companies, right? If you’re running for profit companies, they’re actually giving us more, otherwise it wouldn’t be profitable. It’s in the nature of the transaction. So yeah, we are paying them cash equity, whatever. But the reality is they are giving us more value than we give them. In terms of their capability, skillset, execution, what have you.

So they are customers. Now and this has become doubly so in today’s era, then it was earlier because the demand supply equation has shifted. The market has gone down and all that stuff. There’s a lot more supply in the market, but the truth is that the top 1% that matter, which are the only real skill in the market, 99% of it is fluff, that 1% percent has remained the same forever. And we are all fighting for that 1%. This is how it is because the other 99% if we hire, it’s a net negative. Because product execution is not fun. It’s easy to mess it up. So now you’re competing for a highly saturated market with a very short supply of customers who have lots of options. When you think about this as a product, you really have to say, “I am offering an employment product to the market.” This is what I am doing.

Ajey: Actually, it’s true. Google offers Baristas and chefs as a part of employment products.

Sidu: This is why it is so hard to compete with a Google [-like company], because we say employee experience, but in our minds, when we say employee experience, we are not thinking of it as product experience. In fact, for most people who say the word employee experience, there is no, like for all, when I say product experience, there are images that come up in your mind of things you care about, things that you would do to improve the product. That depth of reasoning is rarely there behind the label employee experience, because we think employee experience is different from product experience. So my argument on this is you’re offering an employment product. What we call employee experience is product experience. You have acquisition channels, you have engagement, you have retention, you have churn. It’s exactly the same. 

So now the question is, if you’re running recruiting, acquisition is operations, fundamentally. You need to know your CAC (customer acquisition cost) channels down like that. Otherwise you can’t execute it. You can’t scale it, right? You lose. Same thing applies to recruiting. You need to treat them like customers. They are demanding customers. Anybody you want to hire already has five offers, for sure. 

You’re also going to be competing against the FANGs (Facebook (META), Amazon, Netflix, and Google) and they’re going to price you out of the market for sure. They’re going to give perks that you can’t match, to your point [Ajey]. 

So now this is true for us on product as well, right? It’s very rare for us to be building a product that is absolutely a first in the market with no competition, with none of that stuff. This is exactly the challenge we take on with our products. I am just simply saying, think about your recruiting the same way with the same level of depth and attention to detail. What is your differentiation? Why is that customer going to pick you over someone else? Why will they stay with you? And the moment you start structuring your recruiting, especially the acquisition part, with the same operational rigor as you run your product acquisition channels, you will start to see dramatic differences. You will also start to do… when you look at product referrals, we spent a lot of time looking at that, there’s a super hardcore metric to run on recruiting, which I recommend for anyone the moment you cross, say, 15 people headcount size, how many referrals are you getting from candidates you have rejected? This is the test of the quality of your acquisition. Those candidates you have rejected are sending you their friends. Because they had such a good experience.

Even potential employees can be your brand ambassadors [30:43]

Ajey: Yeah, we used to have a lot of those kinds of things. Like, we had actually hired somebody after three years [of them trying] in Gojek. He kept applying every six months. We used to say we could not process your resume at this time. We cannot go ahead with your candidate at this time. But please come back after six months. 

That person actually referred us to eight people. And he kept telling people that Gojek is such a great environment. And after three years he came in, when I was quitting, and he actually sent me a message saying, “Finally I am here and it feels good.” So that, and it has nothing to do with… we didn’t have perks. We don’t have many things. What we had was the experience they had the first time. When somebody walks in, when somebody talks, that experience was like… So a lot of time, candidates feel that they are one row in an Excel sheet. You have hundreds of them and you treat each of them equally. One of the most important differentiation is that you’re, it is a hospitality industry. We brought that mindset that we are in hospitality and we are there to help them as much as we can, whether an offer is gone or not, whether they accept it or not, when they leave us, they are our ambassadors saying, “Okay, I did not like them. They did not offer me a good salary, or they did not select me. But overall, I had amazing fun while being at the Gojek office or talking to people.” Whether it is being on time or sending them small gifts. Whether we reject or accept them, we’ll always give them something to take home because if they keep a Gojek cup in their office workspace, there’s a good branding for us, and it doesn’t cost us a lot. So those kinds of things were really interesting and Sidu actually designed a lot of stuff.

Sidu: That reminds me, we designed t-shirts because we realized that t-shirts are a very low cost branding investment. But most people end up doing t-shirts which people wear to the gym or to bed. No one wants to wear it outside. So we discussed this and our head of design basically got into the game and he personally designed it. Like, I still have three of those t-shirts in the wrap. I’ve never taken one. I’m preserving them. But the strategy was this: We want somebody to see our brand in every competitor’s office on hiring. If they’re competing with us on hiring, I want people walking through my competitor’s office to see Gojek t-shirts during the day. I want my brand recall to be there. And we did it. And it cost us nothing.

Ajey: Yeah, we designed very beautiful t-shirts, and they did not carry Gojek in a very in-your-face way.

Sidu: This big (gestures), at the back. That’s it.

Ajey: We had one t-shirt that said, “There’s no one else,” for engineers.

Sidu: So basically what happened there is Abhinit (Tiwari), who is our (former) head of design, used to keep track of all the memes running on Slack. And this was an example of that. Where it’s like, “Look, there’s no one else. Whatever has to be done now, whatever’s on fire now, there’s no one else.” So he used to pull those memes up and churn out t-shirts on them. And those were for internal (people) only.

Ajey: Been there, done that. Got shit done. In two weeks or never. So we had in two weeks [at the front] and behind [the t-shirt] we had ‘never’.

Sidu: Because that was the joke around estimates. Like you say, “How long will anything take?” Everyone is like, “In two weeks.” So we actually had like internal t-shirts for all of this stuff.

On candidate segmentation: One size doesn’t fit all [34:14]

Audience: One question. You mentioned the referral metric, right? What’s the reasonable referral metric to look out for? If you’re interviewing, rejecting or interviewing 100 candidates, how many is a good number?

Sidu: Look, I wouldn’t worry about it that way. The way I would structure it is to say, “What is my number today? And am I increasing it?” There’s no target that I can give you, which makes sense for everyone. I don’t even know how many people you have. My answer to that would depend on how many candidates do you process in a month? For what segments? It’ll be different for say engineering versus operations. So I would say honestly, it doesn’t matter. The question is where is it today? And is it going up? And once you scale beyond a certain number, you want to attribute it to hiring managers. Because you’re like, you will find that specific hiring managers handle candidates in such a way that their rejects bring in more candidates. So at a small scale, that is in the case, you are the hiring manager, but as you scale up, track the metric attributed to the hiring manager.

Ajey: So we did a customer segmentation on candidates, single, married, kids, no kids, 20s, 30s, 40s, two years, five years, six years experience. We would intentionally ask people their history, which country, which city you are coming from, tier-two city, tier-three city. And those were very interesting things. Why did we do that?

Sidu: Yeah. This is right from the get go. We tend to think of the product as the app, but the reality is the product is everything that goes into delivering the value to our customers. Everything upstream. That is actually, if you’ll forgive my slipping on to the technical side of it, the app is just the last artifact in a CI (continuous integration) pipeline. That doesn’t just pop into existence just like that. And something that I think is under-indexed on, except in Googlers who really get it, is that your people policy is a dramatic impactor of the product that gets delivered to the customer, right. 

And you cannot craft [your] people’s policy unsegmented. Too many companies will treat people with kids and people with no kids the same way. And then say, “Yeah, I have some people who just like to be in the office all the time. I’m working so hard. And there are other people who just check out at five,” I’m like, “You understand that the more experienced a person is, the stronger the likelihood that they have kids.” So the moment they’re like, “No, we were hiring, and now we’re scaling up…” So I’m like, “So, you started hiring 30-somethings who are married? They’re like, “Yeah. We started hiring more experienced…” I am like, “Yeah. They have kids. 

You’ve got to structure your people policy differently for each segment. Because they have different needs. How are you going to..?” We were talking about acquisition, engagement, churn, right? How can you engage people if they’re unsegmented? And these are just very obvious things. You do it for your product, right? Age, marital status, dependence. Place of origin and driving out from that first language, because we had three very clear buckets. Native first language, native English speakers, non-native English speakers, like who are not comfortable in English that segmented further. We had primarily Hindi and Bahasa. Hindi in India and Bahasa in Indonesia. Now, if you’re not aware of this, you’re going to make mistakes in how you approach policy, which is going to mess with your engagement, which is going to cause churn.

Ajey: So, when we’re scaling the org, apart from this, what else do you think, in the early days, matters a lot?

Sidu: I talked about this a little bit earlier, but as you start scaling up, what starts to slip through is that mental model. As I said, it becomes a black box. You want to keep thinking about your organization as a meta product. It’s a product that builds your product, right? You, again, you don’t make the car, you make the system that makes the car.

So you, as a founder in the early days, are going to work on the car directly. This is natural. Problem is that as your organization scales up, you continue to work on the car and then the system that is growing, that is actually making the car, because your ratio of input has changed, no. Earlier you were one in three. Now you’re one in five, one in 10, one in 20. Plus you have 20 other things to do in the day. Your input into the product is just collapsing through the floor just because of time. 

But what you’re ignoring continuously is you are trying to…you’re going into harder mode on the product without realizing that you have now actually got to focus on the metaproduct, you no longer work on the product. You work on the metaproduct. You cannot touch the product directly. It’s meaningless. You can, but then you’re doing it for fun. Meaning you’re not doing it to materially contribute to the product. You’re doing it because, okay, it gives you that motivation that, “Okay, I spent half an hour working on this thing, on that thing.” Not because you materially contribute to it. You have to devote the majority of your attention on designing and executing the metaproduct, where the primary visible aspect of it is the people. 

And then there’s this massive invisible aspect of it, which is sitting in inventory in your emails, Jira, Asana, whiteboards, phone calls, whatever, all invisible, which is frankly, something that you should be terrified about. And terrified because also it’s not really tractable. You can’t make it visible. It’s not like a shop floor in a factory where if component inventory is piling up, you walk across the floor, you’re like, “Oh man! There’s something wrong here.” You will have Jira stories piling up and piling up, and you will never know until you go look for it there. You can’t stumble over it by accident. 

So, thinking about the system that builds your product as a product, and the organization as the most visible part of it, but only the tip of the iceberg. Because otherwise what will happen, the other biases kick in. You’ll optimize what you can see. What can you see? The people. You can’t see the work they do. You can’t see the connections between them and most of the heavy lifting happens in the connections between them and the work that they do, and which is why you keep coming back to this default mode where you’re like, “Okay, things are not scaling. I’m stressed out. What level do I pull?” Add headcount. Very natural, counterproductive.

On metrics and scaling data engineering teams [40:53]

Ajey: One of the things which you also did was data engineering. And we actually ended up scaling a very large data engineering organization, which was created literally outside of the main product engineering organization. What do you think in terms of building a very non-product facing organization? What kind of metrics should we use? Because eventually every one of them will either end up creating platform engineering, customer communication, or some organizations which do not directly talk to products at all. Or they do not even have a product interface. They do not even contribute to productivity. What do you think about creating this kind of non-functional or non-directly contributing organizations? And how do you actually justify that? How do you make sure that there is somebody who causes developer experience or platform experience? What does it really mean to create this? And how do you measure the success of it?

Sidu: We were at a million transactions and the existing architecture was really hitting limits. And it was all synchronous. And because there were multiple products, there were all of these integration, synchronous integration points where to deliver something to the app. An example that I like to cite is transaction history. We had at that time, what, like 10 products?

Ajey: 10 products.

Sidu: With two or three high transaction volume products. And when the customer opens the transaction history, there was an API which would call the billing the transaction API for all those 10 products one by one. Like, summarize, put everything together and send it back. Which is all fine, up until you really start to scale, like that stuff is really going to break it. And we knew that at the current trajectory of order volume growth we were just six months away from that and absolutely melting no matter what we do. Like, this entire time we have had a team on it that is just brute force scaling it,  but the fundamental solution is architecture. So these guys decided that, “Okay, it’s time to bite the bullet and go async.” And, I went sufficiently post technical that, while I would spectate and ask questions, I didn’t take the primary decisions. They already had their hands full. He was obviously running full out. Our other co-founder, Niranjan (Paranjape), who was the other CTO, full out. There was literally nobody else. We joked about that t-shirt [earlier], there was nobody else. 

So we needed to be able to migrate to this async architecture. These guys pick the stack, Kafka ‘blah’… Here’s how the architecture of transport, food and other products will change to integrate with… This is the new architecture, these are the pieces we’re going to build. Awesome. And there was nobody else. So I picked it up. I’m still technical enough that I can code but I can’t take these kinds of calls without help. So I said, fine. 

We built a small team. We started executing. Three months later, we had something that was running in a couple of use cases, standard events, and technical leadership was aligned on, “Okay, these contracts make sense to us. Let’s start migrations,” and then we waited. One month passes. No one’s integrated. Nobody. Ajey is losing his mind. His one downs, who are basically CTOs, at unicorn scale of these products are losing their minds. They are giving top down orders to everyone, “Migrate to this stack now. Clock time is running out. This is going to go down.” It got worse. We started having some intermittent downtimes around those subsystems. Like, the transaction is just one, but others started having intermittent downtime. Nadiem (Makarim) started getting into it because the moment downtimes happen in subsystems, Nadiem is like… I’m like, “We are trying to do this!” Nadiem goes out and puts out a message, “This is the number one priority. We need to get this done.” Still nothing is moving. This went on for six months. Things got really bad. And… 

Ajey: We had a major downtime for almost like 12 hours. And we would have downtimes almost every other day for around like two hours. And that day we went down for 12 hours. Like, morning 9 a.m. to evening 10-10.30 p.m.

Sidu: So we were like 120, 130-people at that time in engineering. So I use engineering as a proxy number because you can do the fan out from there. Like for every one or every two or three engineers, there’ll be so many others. So about 120 or 130 engineers and nothing is moving. It’s mission critical. There’s a group CTO mandate, group CEO mandate, division CTO mandate. There is a board of director level executive running the solution. All of us are pushing nothing’s moving. And what I’m about to tell you is the realization we had after organically trying to iterate and crack the problem and look back at what we did. 

So at one point I’m like, “Look, we are failing to distribute. So I asked my colleague, my PM, I’m like, “Look, I want you to split your time. I want you to move 20% into sales.” He’s like, “What do you mean sales?” I’m like, “Dude, we are not picking it up. Now, I want you to go to every product manager on every key product line, because some of those things had multiple products. Like food was not one product. They’re multiple. And I want you to sell.”

And we’ve set up a proper sales Excel sheet. And in that we had like multiple stages, what we call humorously, like the addiction phase, right? Like where you start bringing the product and you run it in trial mode. And then. You can’t go back. You’re locked in. All of that fairly standard CL stuff. Things started moving. Things started moving. They unblocked. The critical products moved over. Of course, there was a longer tail of non critical systems which weren’t hitting limits that didn’t move, but it was moving. Then the larger thing started off, “Okay, now the real burning issues are addressed. Now we have to migrate because otherwise we’re running two architectures. This is problematic. Sooner or later, these will also break.” So then the larger thing started. Again, it was very slow. 

So the next change we made was, by this time, data engineering is like 10 engineers. So we divided 20% into an implementation consulting team. Two engineers on rotation. So every quarter or so, two people move into the consulting team, the other two move back into the platform team. These are names I’m giving you now. We didn’t think about it this way when we were doing it. We were just like, “Look, our customers are telling us that they do not have the talent bandwidth on their teams. Okay. To prioritize this migration, can we solve it by putting our talent over there?” This is how we were thinking about it. 

Then I was like, okay, I’m starting to realize that the stuff I’m doing over here is B2B SaaS. Look at what changes we have made. Now I have formal sales, which have now gotten stuck. Like, we actually had a formal sales pipeline. And now I had two PMs, one of the PMs owned a formal sales pipeline perpetually that did not die there. That became a standard. Second thing, I have an implementation consulting team. Then I’m like, and this is the mental model, so I’m like, “Okay, guys, let’s think about it like this. I’m a founder. I have a relationship with Nadiem and Ajey. But we are not in the company, we are outside. So I’ve gone and done a handshake deal with him. But now we’ve got to get seats. I’ve got to sell. This is how we need to think about it.” So then we started setting up, literally what today, in today’s world, would be called a DevTools SaaS website. Wiki. 

We made the documentation pretty. We had tons of documentation. All the READMEs were there, because obviously it’s DevTools. Pulled it up on the website, made it searchable, made it pretty product pages, product lines. 

Once we really went all the way into this model, “Boss, this is a B2B SaaS company that’s captive, but it is nevertheless a B2B SaaS company,” things started moving. After that every subsequent, what you would call a platform play, in any other company, we simply modeled it as a captive B2B SaaS. Then we started realizing that as you really scale up, this also makes it easy for leadership to have levers to control because what you are literally doing is saying, “Look, I have a bunch of captive companies. I have captive services.” You don’t have functions anymore. You have a captive design firm. You have a captive PM firm. You have a captive engineering firm. Then you have all these other platforms, which are captive product vendors to your consumer team. So your consumer products are now buying services internally. You give them a budget. They go to the engineering services firm and they’re like, “Look, I need this kind of talent at this…” And the engineering firms are like, “These are the price points.” 

And eventually we realized, again, I’m saying all of this is us looking in hindsight and structuring what we organically stumbled in. Gojek India was a captive services firm. So the moment we shifted to this, economic control levels became clear. So to address Ajey’s question, the platform stuff, it’s a captive B2B SaaS, make sure it has P&L (profit & loss). That was the last stage that we never managed to get to because of IPO-readiness, finance team… Otherwise we were all ready to go to a level where it was like boss, “I will invoice you because I need to know how my product is performing. Yeah. I won’t invoice you outside, but I will invoice you inside because that’s our contract.”

Ajey: And the reason we went up to that extent is because a lot of times you don’t see the value of SRE (site reliability engineering) teams. And this is what is going to happen to all of you in your SRE teams. Because they will try to roll out new architecture, new VPCs (virtual private cloud), new container models, new databases and nobody is going to move.

Sidu: Because they have their own OKRs, right?

Ajey: Because everybody has their own OKRs. We actually did that and eventually that made our internal documentation so amazing that people actually started calling it, now people call it something like an internal open source, they have a word for it. We actually had internal open source for a while, where people can actually access any repository, contribute to it. Every service had an SDK (software development kit). Anyway, those are more interesting details. We shouldn’t get into that. Let’s stick to the org and recruitment and dealing with the PMF. We have five, seven minutes. Any questions from you guys?

Optimizing collaboration: A developer’s perspective [51:26]

Audience: So you mentioned links going wrong. Like how do we think about making sure that they go right?

Sidu: Can you [clarify] which links? Links between people?

Audience  Links between different teams… 

Sidu: Yeah. So I’m not sure that my approach would work for everyone because my original story is that I’m a developer. So I’m calling that out. But for me, I look at an organization as a distributed computing network, straight up. In fact, I don’t just look at the org, I look at the entire system as a distributed computing network where I have some nodes which are humans and some nodes which are machines. The whole point is highly repeatable processes which can be scaled are delegated to machine nodes. Non repeatable, creative, values-driven decisions are delegated to human nodes. And you can instrument and observe the system the same way you would observe any network. What is the latency? What’s the band, what’s the throughput? Is there packet drop? Are there errors? It’s just that, with human nodes, you know that the packet drop is extremely high. Serialization, deserialization are extremely error prone. It’s extremely lossy. So you want to look at subnets within that which is what you would call a team. And you want to then look at the interfaces of that subsystem and say, “How is that subsystem communicating with others?” So you can look at it individually, from one person to another in a team. 

So you consistently see miscommunication between two. Before I started thinking like this, I’d be like, “Why can’t you people just figure it out? Why can’t you just…” None of that is actionable input as a manager, right? Instead I would start digging into, “Okay, why is there packet drop here? Are these people remote? Are they in different countries? What are the processes they use?” Like, one of the most powerful things we did, another colleague of ours introduced into the organization [something called] silent meetings, which dramatically reduced packet drop and loss. So there’s a lot of process that you, so your interventions have to be operational. This mental model will allow you to instrument and discover problems. 

But it’s very important to not slip into your intervention being, “Try harder!” Intervention has to be operational. There’s a process change that you bring in and that has to fix the issue. Between teams, again, you just look at it throughput, drops, errors in communication. Then you diagnose it. But the point that I made [about] links being invisible, is just to address the bias because we will see the individuals, but what is actually at play there is our organization. The organization is invisible. Only the individuals are visible.

Ajey: Thanks a lot, Sidu, for actually coming and helping us. 

Sidu: Thanks a lot for having me, Ajey. Thanks everyone. 

This transcript has been edited for clarity.