Lets stop Over Engineering

Most of the young programmers will love to work around lots of layers components. And will love to introduce lots of different components. I have seen programmers with 3-4 years of experience wants to work on MySQL, Redis, and queuing servers for different types of the task they were assigned. It is not that they should not, it should be the question of whether it is a must to introduce newer components.

Once my friend was building a feature for billing payment. After the payment is done successfully, he has to do the following list of things

• Updating the license from one to another
• assign freebies like in Gpay, in email
• send a payment confirmation email notification

The total number of requests which came to his app server was around 5K (5 years back), and now it's around 15K per day which is around 10 requests per min. This request count is considered to be relatively very small in number.

The old flow was like once the purchase is done, the license will be updated then freebies are distributed and an email notification is done and finally, a payment confirmation email is sent all are done sequentially.

My friend, who was very young around 3+ years of experience, came up with one of the good solutions where he has to introduce the new component to the existing system. He designed using the messaging server. Since I never had enough exposure to those messaging systems, I asked him whether it is necessary to build using this complex message system when this can be done using CRON jobs and MySQL (the system was already using CRON jobs for different tasks). But later his proposal was moved forward,

At the end of the day, his aim to learn new components was fulfilled, but he missed one important aspect of building the system; whether it is startup or enterprise we should design our system in such a way the throughput is maximum. We should never under utilize the resource we have got, we have to make sure it is used properly. Estimating the hardware cost is very important in the company. That is one of the ways engineers can add value to the company.

When we engineers bring the solutions to the table, it's our duty to understand the trade-off and not over engineer the system. Sometimes the simple quick solutions can also work for next 3-5 years. In my friend case the below solution might have worked properly for 5 years as the requests are not so huge.

The easier solution (without Messaging queue)

• Once the payment is made, the status should be updated in the MySQL.
• Next we have to maintain a table where one time jobs (CORN jobs) has to be done on certain profile which can be run periodically (we can run for every 1 min)
• One freebies are sent, we can update the profile with status.
• Then in next batch, the payment notification mail can be sent

In the above solution we are not introducing new system which can easier the job of some developers; and also the new component is not needed for long time (3+ years).

The only trade-off in the above solution is what if the number of requests increases so huge 500K in a single day, on those times we have to think and introduce the components accordingly. We can for sure understand the trend of growth.

I will be happy to learn in the comments if any other easier solutions can be derived. The suggested solution is based out of my experience and its not implemented anywhere before.

Note: All the image are owned by the service providing companies; the banner image is designed with help of Canva