Discussing how exceptions can be handled in a centralized spot and how to return consistent responses. Exception logging is also centralized in the same approach. — UPDATE April 10, 2022: all projects in the GitHub repo have been upgraded to .NET 5. Motivation I adopted a REST API project today which had no exception handling or logging, making it really hard to investigate issues. So I spent an hour adding a centralized exception handler and structured logging…

Discussing a GitHub starter project to build a web API using Partitioned Repository Pattern with Azure Cosmos DB, ASP.NET Core, and Clean Architecture. — UPDATE April 10, 2022: all projects in the GitHub repo have been upgraded to .NET 5. This article describes the GitHub project that can be used as a starting point to work with: Clean Architecture (Onion Architecture) ASP.NET Core 3.1 Azure Cosmos DB .NET SDK V3

How to deploy a trained sentiment analysis machine learning model to a REST API using Microsoft ML.NET and ASP.NET Core, in just 15 mins. — Why fast integration? If you have a pure Data Scientist role, you probably care more about studying the data, going through feature engineering, choosing the correct statistical or machine learning model, and training the model to perform well on both the training dataset and validation dataset. And that’s it!

Discussing how to scale the barebones TinyURL system using Load Balancer and Database Horizontal Partitioning, and related topics like partition key design strategy, partition vs replica vs backup, SQL vs NoSQL. — In the last couple of articles, we discussed how to design a barebones working solution for TinyURL, and also how to improve 90% of the estimated workload by introducing cache. The system diagram below shows what we currently have. The single web server handles all the HTTP requests from the…

Discuss how to improve a sequential workflow to call external services with multi-threading and one possible approach to handle the thundering herd problem using a thread-safe collection. — Background We often have to call a same external service multiple times, or multiple external services, in order to get all the data we need in order to process the data. The easiest way to handle this is just to sequentially call the external service(s), and process the data afterwards. For…

Part 4 : Discuss how to use a separate Console Application as our consumer instead of an in-process consumer so it can easily be scaled out to multiple instances. — Background In the last several articles, we have designed a smart coffee machine system that would decouple long-running tasks, i.e. making coffee, from the regular HTTP request processing. We have also taken our iterative approach to build a solution that implements the design. As you can see from the system diagram…

Part 3: Discuss how to use Azure Service Bus as a message broker instead of an in-memory message broker and how topics and queues are auto-configured. — Background In the last couple of articles, we designed a smart coffee machine system that would decouple long-running tasks, i.e. making coffee, from the regular HTTP request processing, so that the user would get a response right away; we also took a very first stab at implementing the smart coffee machine…

Part 2 : Discuss the simplest implementation to decouple long-running tasks by using an in-memory message broker and an in-process worker that completes the tasks asynchronously. — Background In my previous article, I built upon the Microsoft documentation on ASP.NET Core Performance Best Practices, and discussed how long-running tasks could be decoupled from the regular HTTP request processing in a REST API application at a conceptual / system design level. We also came up with a system design…

Discuss how to choose and design a database for telemetry data sent from IoT devices in a smart home. — Background Smart home is not a new concept anymore, especially to folks who are into technology. The most common smart devices in a smart home can be speakers (e.g. Apple HomePod, Google Home, Bose Sound Touch), thermostats (e.g. Ecobee), lights, cameras, etc.. In fact, if you have a phone app to…