Here at RV, we’re proud to work with many talented, innovative tech professionals who power our tools and platforms – including Majid Fatemian, a Principal Engineer on our Red Platform team. As he shared in his segment at our bi-annual RV RUN. Tech Summit last fall, many of us – regardless of whether we work in tech – often face crossroads in our projects, where we stumble upon an unexpected obstacle and have to decide whether to stay the course or try something new. Whether you’re a tech superstar or a project manager or a designer or anything in between, Majid’s tips are here to help you navigate those crossroads with confidence.
The Unknown Unknowns
When we start a project or begin a new functionality within an existing system or a migration, we do a proof of concept. Everything looks great at the beginning! But, as we progress into the details, we begin to face unknowns that we never considered. We learn that not everything goes according to plan, and, unfortunately, that proof of concept doesn’t always happen word for word.
This is when the challenges start to creep up, one after another… Finally, we have to decide: Pivot or Persevere?
Should we push through the challenges and get over them? Change directions or make a U-turn?
In this article, I will share what I’ve learned when faced with these crossroads. I’ll share how my team made difficult decisions and, hopefully, it will help you make yours.
Tip 1: Speed up your FEEDBACK loop
In the book The Lean Startup, Eric Ries says:
“The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop.”
In my opinion, a new project, functionality, or migration of systems is not much different. The shorter the feedback loop, the faster the team can decide whether to pivot or persevere.
It is best to make the experience as close to a real-world example as possible. That’s why contrary to popular belief, I encourage teams to…
BONUS TIP: Go into the production environment as early as possible.
The sooner we start dealing with real-world scenarios, the sooner we spot all of the hidden challenges. In several instances working with engineering teams at RV, our projects went smoothly until we started dealing with the production volume of data. But, if we had dealt with the production data sooner, it would have been easier to catch the problems and progress from there.
Tip 2: ZOOM OUT
Technical challenges can be very blinding. We see one challenge and try to come up with one solution. However, as soon as we find the solution, it may cause another challenge, and then we’re busy solving the new one. Over and over again. We get pushed down a rabbit hole that is so far diverged from the primary goal.
Tech Case Study
In the case of migrating our Identity Graph to a Graph database, we couldn’t leverage our existing tech stack. Most of our tech stack is in Go, but the chosen graph database engine didn’t have an official Go toolset, and the unofficial ones were having problems that we had to roll out.
Hence, we used Serverless AWS Lambda functions to read from the main Kafka stream, processing and pushing to the Graph database engine.
After that, we faced another roadblock, with the inability of Lambda trigger for Kafka to scale more than the number of partitions in the stream.
To overcome that, we delayed the processing to another step. We created a Lambda function that reads from the Kafka writes to an SQS queue and fan-out there so the Lambda function can read off that queue. SQS and Lambda scale much better together. To overcome that, we delayed the processing to another step. We created a Lambda function that reads from the Kafka writes to an SQS queue and fan-out there so the Lambda function can read off that queue. SQS and Lambda scale much better together.
At the time, we could not see the monster we were creating. We were adding so much architectural complexity to our system, when our real goal was to simplify our codebase.
This is why technical challenges are sometimes blinding. We get hit by one, try to solve it, then the next one, and so on – death by a thousand cuts. Results contradict the goals.
If we zoom out at each step, look at our entire roadmap, and verify where we are versus what our final goal is, it will be a lot easier to see the problem we’re creating, and move past it.
Tip 3: Ask for HELP!
In difficult situations like this, you are not alone. You don’t need to make the decision all by yourself. It’s a team effort. Even if you’re a team of one, you have external resources that can help you – like this blog post! 🙂
Search for experts within your organization. Others might have gone through a similar path, and their perspectives could help and facilitate decision-making.
TECH TIP: If you have vendors for services like Cloud computing, they usually offer assistance and consultation. Because they have exposure to so many customers and clients, their insight is significant and extremely valuable.
The tech community is usually very supportive. Reach out to experts and peers for feedback.
Tip 4: COMMUNICATE
As with most situations in life, our challenges are usually miscommunication problems. The same thing applies to engineering and software engineering. I believe that software engineering is more about COMMUNICATION than coding.
In situations where we feel stuck, communicating early on with leadership and production teams will help them to be aware of the problem. Then, later on, when more challenging decisions need to be made, nobody is surprised. Also, the process of decision-making becomes collaborative, leaving less room for error.
Technical leadership, product management, and project management all add different perspectives to situations that can help ease decision-making.
Tip 5: Let the PRODUCT be your ultimate guide!
All of your efforts are related to the product in some way or another – whether you are enhancing capabilities and performance, adding functionalities, or using a better service or technology.
So, the product itself is your best guide. Look up to the northern star!
Ask yourself: Are the efforts…
- Adding business values?
- Reducing the technical debts?
- Aligning with what the customers are asking for?
- Within the required or desired Service Level Agreements?
Tip 6: Both pivoting and persevering can lead to SUCCESS
We should not consider a pivot a failure. When we are too engrossed in a technical challenge, we sometimes fall into the trap of cost fallacy. We have spent so much time and energy on the topic that it seems like a waste to pivot. We have invested so much into it. But, we must consider that stopping the waste and the bleeding is, ultimately, a success. It’s never too late to mend.
Document your processes to help you remember why you decided to pivot or persevere. We tend to forget the reasoning behind our choices, which may cause us to fall into a similar trap again. Documenting the decisions you make is essential to avoid wasting time trying something that you have already tried and tested.
In conclusion, roadblocks are inevitable. There will be many times when you have to decide between pivoting or persevering. But, with the approaches above, you will be able to spot the roadblocks before getting too far down the road, and, hopefully, you will feel more equipped to make a successful decision!
Interested in diving deeper into Majid’s piece? Read his in-depth version – featuring helpful tech case studies – right here on his personal blog.
TECHnically, you don’t have to stop reading here – get even more tech tips from Principal Engineer and fellow RUN. Tech Summit speaker Chase Coney right here.