Discussing how exceptions can be handled in a centralized spot and how to return consistent responses. Exception logging is also centralized in the same approach.
Motivation
I adopted a REST API project today which had no exception handling or logging, making it really hard to investigate issues. So I spent an hour adding a centralized exception handler and structured logging using Serilog. …
Discussing a GitHub starter project to build a web API using Partitioned Repository Pattern with Azure Cosmos DB, ASP.NET Core, and Clean Architecture.
This article describes the GitHub project that can be used as a starting point to work with:
- Clean Architecture (Onion Architecture)
- ASP.NET Core 3.1
- Azure Cosmos DB .NET SDK V3
- Repository Design Pattern
- Partition Key and Partitioned Repository
Popular features also supported in the project include:
- Cosmos DB database initial…
How to build an ASP.NET Core API endpoint for time series anomaly detection, particularly spike detection, using ML.NET to identify interesting intraday stock price points.
Background
What is Anomaly Detection?
According to Wikipedia, “In data analysis, anomaly detection (also outlier detection) is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data.”
In plain English, anomaly detection identifies weird behaviors or data points, such as power outages, email fraud, viral…
How to deploy a trained sentiment analysis machine learning model to a REST API using Microsoft ML.NET and ASP.NET Core, in just 15 mins.
Why fast integration?
If you have a pure Data Scientist role, you probably care more about studying the data, going through feature engineering, choosing the correct statistical or machine learning model, and training the model to perform well on both the training dataset and validation dataset. And that’s it!
However, in most cases…
Discussing how to scale the barebones TinyURL system using Load Balancer and Database Horizontal Partitioning, and related topics like partition key design strategy, partition vs replica vs backup, SQL vs NoSQL.
In the last couple of articles, we discussed how to design a barebones working solution for TinyURL, and also how to improve 90% of the estimated workload by introducing cache.
The system diagram below shows what we currently have. The single web server handles all the HTTP requests from the…
Discuss how to choose and design a database for telemetry data sent from IoT devices in a smart home.
Background
Smart home is not a new concept anymore, especially to folks who are into technology. The most common smart devices in a smart home can be speakers (e.g. Apple HomePod, Google Home, Bose Sound Touch), thermostats (e.g. Ecobee), lights, cameras, etc.. In fact, if you have a phone app to…
Discuss how to design and complete long-running tasks outside of HTTP requests in RESP API, as recommended by Microsoft on ASP.NET Core Performance Best Practices.
Background
Most applications have some tasks that take longer than normal to complete. These tasks can be to generate a complex ad-hoc report for downloading, to trigger and wait for a CPU-intense computation, or to perform a series of small tasks that add up the whole processing time. Although there is…
Discuss how to add a message producer to a distributed messaging system using REST API with ASP.NET Core and publish messages to the message broker using RabbitMQ.
Goal
In a previous article, we built a distributed messaging system where a message broker using RabbitMQ and message consumers using ASP.NET Core hosted services were set up and running. In this article, we will discuss how to add a message producer using ASP.NET Core REST API to the messaging system…
Get Redis, SQL Server, MongoDB, and more up and running in docker in 15 mins!
Background
Let me start by asking and answering a few quick questions:
- Have you installed a full SQL Server on your dev machine? Yes, I have.
- Have you installed Redis on your dev machine? Yes, I have.
- Have you installed all the other databases that you temporarily need on your dev…
Discuss how to quickly setup a distributed messaging system, including consumers and RabbitMQ with Docker in ASP.NET Core, hopefully, in less than an hour.
Background
What is distributed messaging system? A distributed messaging system allows asynchronous processing of messages or workloads in microservices architectures. Such a design decouples the message producers, message consumers and the message transport, allowing the design to be highly scalable. Amazon documentation does a great job explaining the basics. …
