How to use the .NET Core 3.x REST API Template for Visual Studio?

.NET Core has become a very interesting framework to develop with. Version after version, it has grown and included significative improvements. However, more often than not, those changes were translated into repetitive changes (in APIs, libraries, dependencies, possibilities, the way of coding, etc) which became time-consuming and frustrating.

Two ways to install the extension:

  • through Visual Studio
  • through the Visual Studio Marketplace

Visual Studio

To install an extension from Visual Studio, go to Extensions -> Manage Extensions. Once there, look for “rest api” on the Online/Visual Studio Marketplace section.

VSExtension

Follow the instructions after clicking on “Download”.

Visual Studio Marketplace

You can download the extension from the Visual Studio Marketplace.

MarketPlace2

This is the direct link to the extension.

VSExtensionMarketPlace

The “Download” button will start the download of the VSIX file. Once downloaded locally, double-click on the file to install it and follow the instructions.

How to create a new solution with the template?

The creation of a new solution based on the new template is quite straightforward. The process is identical to any other creation of a project/solution. This was important to provide good user experience.

First step, open Visual Studio.

NewProject

Click on the “Create a new project” button.

NewProject2

Choose the template and click on “Next”.

Note: if you do not find the template, use the search field or the filters on the top of the window.

The window after will request common information for new projects/solutions.

NewProject3

The process will take a couple of seconds and it will create a Visual Studio solution with the structure below.

Solution

Description of the created solution

We now have a brand new solution which already includes valuable items:

  • a controller illustrating how to use it (UserController).
  • layers separation in terms of responsabilities.
  • integration of Unit Tests (controllers and REST flows).
  • DI and IoC integration based on .NET Core 3.1 practices, with no external dependencies.
  • mapping configurations among layers (based on Automapper)
  • OpenAPI configuration with authomatic generation according to code and comments (XML comments).
  • OpenAPI automatic versioning according to code and tags (versioning based on URL routes but editable if needed).
  • exception handling according to .Net Core 3.1 practices.
  • implementation of all the layers except the repositories (hard to generalize nowadays a repository type). Not even sure your API needs one.
  • minimal dependencies.
  • condensed of common best practices in terms of REST API coding.
  • an already running API with automatically created OpenAPI, which makes things much easier for developers starting with all these concepts.

Let us go a bit further and understand the main concepts.

Structure of the solution

The solution is organized according to the conceptual layers below:

  • API
  • Services
  • Repositories

The solution contains a folder for each of these layers.

In addition to these folders, the solution includes a Tests folder and a Common folder, including factorized components.

All that looks like this:

Folders

Note: the numbers of the folders are set to order them in a practical way.

Folders

1.API

The API folder contains the entry point of the project.

API

API:

  • SolutionName.API (entry point, Startup project)
  • SolutionName.API.Common, factorized components at API level
  • SolutionName.API.DataContracts, exposed data types

2.Services

Services

Services:

  • SolutionName.Services, implementation of the business logic of the solution

3.Repositories

This folder is empty on purpose. It is created to anticipate where to create your repository (in case you need it).

No repository implementation has been provided because there is a wide variety of persistence models. Additionnaly, your solution maybe does not need any.

00.Tests

Tests

Tests:

  • SolutionName.API.Tests.Controllers, testing the controllers directly
  • SolutionName.API.Tests.REST, testing the rest routes through REST data flows

01.Common

Common

Common:

  • SolutionName.IoC.Configuration, DI and Automapper configurations
  • SolutionName.Tools, different types of Tools or Helpers

Projects

All in all, the solution creates 8 projects, named, organized and referenced according to a certain set of practices.

API:

  • SolutionName.API (entry point)
  • SolutionName.API.Common
  • SolutionName.API.DataContracts

Services:

  • SolutionName.Services

Common:

  • SolutionName.IoC.Configuration
  • SolutionName.Tools

Tests:

  • SolutionName.API.Tests.Controllers
  • SolutionName.API.Tests.REST

All this may seem a bit overwhelming. Let us understand what are the principles followed and all will make more sense.

Principles

The whole template follows basic architectural and developing principles.

The layers are separated conceptually according to their roles and responsibilities.

Every layer exposes its entity types through given contracts. Those contracts are implemented and resolved by DI. The entities among layers are mapped. One entity type from a given layer should not be used in another one.

The API folder includes a separated project for each layer. This makes things really clear conceptually and phisically at this level.

Services folder includes only one project merging Contracts, Models and Services. Being puristic and rigurous, each of them should be implemented in its own project. That would add another 2 projects, maybe complexifiying the solution too much. This separation is recommended though in projects of a certain size. For small projects, you will maybe have the feeling of “over-architecturing” (at least, physically). But the conceptual practices are all followed all the time and in the same way.

IoC and DI are factorized in a common project (Common folder) to use exactly the same settings in the application and in the test projects. This enables unit testing with closest configurations to real execution contexts.

Dependencies

The dependencies are set automatically by the template.

The external dependencies or additional packages have been avoided (when possible) in order to minimize the dependencies. This reduces the size of the final package.

Controllers

Controllers are the “exposed” part of the whole REST API solution.

They contain the implementation of the different REST endpoints. Those endpoints expose data contracts (conceptually speaking).

The controllers rely on services to gather data or operate actions.

Mappings

Mapping entity types between layers is not the most funny coding party. Automapper helps us with this.

DI and IoC

The DI settings have been placed out of the projects to factorize them and reuse them.

This is an important point, mainly for the test projects. Indeed, this practice allows to load exactly the same settings in different projects (for instance, unit tests and the actual application).

Unit Tests

The projects included in the solution focus on two aspects:

  • testing the controllers
  • testing the REST endpoints

In other words, the test projects will test the exposed surface of the project.

This tests may be completed with other projects to control the quality at the Service level.

How to add code and logic to the solution?

Created code by default

Once the solution is created, you can run it.

The url http://localhost:port/swagger shows the Swagger UI (OpenAPI) with the whole description of the API (dynamically generated according to the code and XML comments).

Swagger1

The versioning is automatically generated too.

SwaggerVersioning

The data contracts are described according to the code definitions and constraints.

SwaggerTypes

The created UI allows to go down into the details of each endpoint (response types, media type, parameters and their types, etc). You can even test the endpoint from the UI.

SwaggerEndpoint

All the information, comments and descriptions come from the code, which makes an interesting way to document APIs in a very productive way.

Your code

To add your own code, you should respect the principles of the template.

Find and follow your own way to build and separate the controllers. Avoid creating only one controller containing endpoings of all types.

For instance, we will add an endpoint to get a car by its plate number.

For this, we will need to follow the steps below.

Services

  • create a Car class in the Services project (Model project folder)
  • create the ICarService interface in the Services project (Contracts project folder)
  • create the CarService class implementing the ICarService interface in the Services project (Services project folder)

CarService

API

  • add a data contract Car to the DataContracts project
  • add a controller to the API project (in the folder corresponding to the right version, let us say in the V1 project folder)
  • implement the GET endpoint

CarAPI

 

Common

DI

Register the service (Configuration project, DI project folder, ServiceCollectionExtensions.cs file)

public static void ConfigureBusinessServices(this IServiceCollection services, IConfiguration configuration)
{
if (services != null)
{
services.AddTransient<IUserService, UserService>();
services.AddTransient<ICarService, CarService>();
}
}

Mappings

  • configure the mappings between the service entity and the data contract entity (Configuration project, Automapper/Profiles project folder, APIMappingProfile.cs file)

public APIMappingProfile()
{
CreateMap<DC.User, S.User>().ReverseMap();
CreateMap<DC.Address, S.Address>().ReverseMap();
CreateMap<DC.Car, S.Car>().ReverseMap();
}

At this stage, you can run the REST API. If you load the OpenAPI (former Swagger), you will see the new endpoint.

CarSwagger

If you expand one of the endpoints (both are identical but one is accessed through the “V1” route and the other one by a “default route”. This behavior can be changed to avoid default routes.), you can test it.

ManualTest

Unit Tests

  • add the controller level unit test in the Controllers project (ControllerTests/V1 project folder)

CarUnitTest

And that is all.

Conclusion

You will maybe have the feeling you need to keep in mind too many things or you have too much work to accomplish for adding simple things.

When projects grow or if you need to create REST APIs in an industrialized way in teams of developers, you will probably see things differently ;).

 

How to use the .NET Core 3.x REST API Template for Visual Studio?

Service Principal for Azure

Azure DevOps, pipelines, releases, Service Principals and Azure…..a nice mix that can sometimes become tricky to configure and to understand what is required to make it work properly.

Context

Releases on Azure can cover fields like:

  • create the target platform (IaC)
  • configure permissions and security (RBAC)
  • deploy the applicative packages

These tasks require an Azure DevOps Service Connection. One of the ways of creating one of them is through Service Principals.

If you create your Service Principal in a regular way (Azure Portal, az ad sp create, etc), you will probably need to check the settings in the next section.

These settings being quite common, the Azure CLI team has created a command to help with all this:

az ad sp create-for-rbac

Settings to check

In order your Service Principal has the right configuration to work with RABC configurations, you need to check the settings below:

  • At AAD level
    Check the Service Principal API Permission has an AAD MS Graph (read all), application mode

    AAD_DevOps_SP_MSGraph

  • At Azure Subscription level
    At IAM settings, check the Service Principal has a User Access Administrator role assignmentAAD_DevOps_SP_IAM

That’s all :).

 

 

Service Principal for Azure

dotnet new webapi3

What is “webapi3”?

webapi3 is a .NET CLI template that creates a REST API solution with the following characteristics:

  • business-oriented
  • layered and multi-projects (C#)
  • following different types of architectural patterns
  • based on .NET Core 3.1 and the last available dependencies
  • minimized dependencies
  • DI, IoC, Automapper, configurations factorization, unit test projects, exception handlers, traces and loggers
  • automatic Swagger / OpenAPI generation (configurable)
  • automatic API versioning with the related Swagger / OpenAPI updates
  • can easily be extended or adapted due to the architecture

 

What is it for?

In the end, the extension is a time-saver tool allowing focusing more attention on business sides of the development and creating value faster.

 

Why?

I guess there is not only one single “why” but a set of different “whys” that end up justifying why this extension has been created.

These could be many of those whys:

  • the gap between the default templates and those required by the real-life developments is too important
  • developers do not like wasting time in repetitive tasks with no real value (mainly if those tasks could be automated)
  • saves time and allows to keep that time (and energy) to focus on creating real value
  • provides homogeneity in team development. This belongs a way of controlling certain types of risks when senior developers are constantly overwhelmed and do not have time to monitor how teams work.
  • Eases juniors’ integration in a team

 

How to install it?

dotnet new -i|–install [package_name]

More details here.

How to use it?

dotnet new webapi3 -n [solution_name]

More details here.

VSIX equivalent

This .NET CLI package is based on the VSIX version.

dotnet new webapi3

.NET Core 3.x IoT Hub REST API Template

After the upgrade of .NET Core 2.x REST API Template to .NET Core 3.x REST API Template (preview first with RC dependencies and final release later), it was the turn of the .NET Core 2.x IoT Hub REST API Template (formerly published as “.Net Core IoT Hub REST API Template”) to be upgraded to .NET Core 3.x IoT Hub REST API Template.

Here it is :).

What is this extension for?

This extension follows the same principle as the others: help the developers to go faster when it comes to building structured solutions.

In this case, the extension creates a whole Visual Studio solution of a REST API (.Net Core 3.1) exposing an extense part of the Azure IoT Hub C# SDK. The solution relies on the .Net Core 3.x REST API Template principles and it could be a good starting point to create running APIs.

That said, the template includes certain simplifications that could maybe require to be reviewed or adapted in your own context.

Why to build a REST API to expose IoT Hub features?

There are many reasons explaining why building a facade in front of an IoT Hub. The most common could be:

  • security
  • additional business logic

 

What does it look like?

Once the extension is installed, you can use it directly from the start screen when you create a new project:

IoT Hub REST APIs Templates
Start screen – New project

You only need to:

  • pick up the template of your choice
  • chose the solution folder
  • give a name to the solution

The extension will create the solution and the projects to build the REST API.

IoT Hub REST APIs Templates2
Visual Studio solution

 

Before running the solution, you need to provide the IoT Hub connection string in the appsettings.json file, in the API project.

IoT Hub REST APIs Templates5
Settings

 

Basically, that is all. After that, your solution is ready to be used. You will find below the published OpenAPI.

IoT Hub REST APIs Templates3
OpenAPI

One of the very practical features is the automatic API and OpenAPI versioning.

IoT Hub REST APIs Templates4
Automatic API and OpenAPI versioning

Of course, most of the time you will need to make changes and adapt the solution to your own context.

I hope you will find all these extensions helpful and useful.

.NET Core 3.x IoT Hub REST API Template

.NET Core 3.1 REST API Template for Visual Studio and .NET Core CLI – Release (VSIX/NuGet)

Full release based on .NET Core 3.1

Many months after .NET Core 3.0 was announced, the release version of “.NET Core 3.x REST API Template” has been published.
All the dependencies have finally been rolled out in a stable version. The “.NET Core 3.x REST API Template” has been updated as soon as possible right after all dependencies of .NET Core have been published in release versions.

Visual Studio VSIX

You will find here the VSIX for Visual Studio.

NuGet version for .NET Core CLI

A NuGet version has been created to make the template usable through .Net Core CLI.
You will find it here.

What is this template for?

Another post in this blog describes what this extension has been created for.
You will find here a short reminder:
  • save time
  • be more efficient
  • help you to build business-ready REST API architectures
  • get rid of repetitive and automatable tasks
  • homogenize architectures and code
  • take advantage of many .NET Core features, fastly and reliably
  • minimize dependencies
  • avoid depending on technical skills to build the kind of provided solutions
  • accelerate your production

 

.NET Core 3.1 REST API Template for Visual Studio and .NET Core CLI – Release (VSIX/NuGet)

TF31003 error when connecting Excel 2019 to Azure DevOps

 

TF31003

 

If you are having a TF31003 error when trying to connect Excel 2019 to Azure DevOps,  these are the steps to follow to solve the issue.

Context:

  • Windows 10 x64 Enterprise
  • Visual Studio 2019 Enterprise
  • Excel 2019
  • Azure DevOps

 

Solution:

  • Open Visual Studio Installer
  • Go to Individual components tab
  • Uninstall Office Developer Tools for Visual Studio

OfficeDeveloperToolsForVisualStudio

OfficeIntegration2019

  • Install it

 

And that is all ;).

 

 

TF31003 error when connecting Excel 2019 to Azure DevOps

.NET Core 3.x REST API Template

Note: the template has been upgraded to .Net Core 3.1 with all the dependencies in the last available release version 🙂 (January 2020).

.NET Core 3.0 has just been released.
.NET Core REST API Template could not miss such an upgrade. So, here it is :).

This new version is not only a technical upgrade. It includes:

  • DI according to .NET Core 3.0 practices
  • dependencies minimization
  • better XML comments to go deeper in OpenAPI/ Swagger specifications
  • “application/json” default setting
  • global error handling and logging with the appropriate code for implementing two different solutions:
    • middleware-based
    • lambda method-based
  • small fixes

For now, the .NET Core 3.0 version of the extension will remain in preview because of the dependencies that have not released in GA.

That said, the extension is fully functional and you can start using it.

.NET Core 3.x REST API Template

Azure IoT Device Simulator Vs Microsoft Device Simulator (Accelerator)

IoT Development

Working on IoT global solutions is very exciting by the number of topics involved: functional valuable features, business value, devices, data, security, data analysis, AI, etc

Testing the solution with real devices is not always efficient and not even possible. So, sooner or later, IoT developers are confronted with the problem of needing devices but not having real ones.
This is where IoT developers start building their own applications or “IoT Simulators”. This is also  where I see a lot of people (developers) coding all more or less the same thing in order to test their solution …. or if not exactly the same thing, at least similar enough to start wondering: “wait a minute, wouldn’t be helpful taking the tool I created for myself, clean it a bit, make it at least ‘deliverable’ and share it?” :).

 

IoT Simulators in the “market”

Many IoT solution providers or editors have provided:

  • pieces of code simulating partial behaviors of devices
  • small solutions but not covering all the important functional flows
  • whole simulation platforms (more or less customizable)

Most of them remain focused on data generation.

However, I still felt the need to go further and build a whole IoT simulator covering most common IoT use cases. This means data generation but also other C2D and D2C flows.

This led me to build a tool for my own needs that has become Azure IoT Device Simulator now.

 

Differences with Microsoft Device Simulation Solution Accelerator

 

In the last years, I have been working with Microsoft’s ecosystem. So, the rest of the post will be related to Microsoft technologies.

Microsoft understood the need for simulating devices quite soon and published:

The simulation platform is a web solution that goes quite far and allows defining:

  • device types
  • device messages
  • emission intervals
  • a few Cloud To Device (C2D) interactions (Direct Methods, as Olivier Bloch and Troy Hopwood present here in the IoT Show)

For now, the main focus seems to be data generation though. Microsoft’s IoT Teams release updates very often. Keep posted to see how the solution evolves because I am pretty sure they will complete the solution iteration after iteration.

Azure IoT Device Simulation follows another approach and tries to cover more IoT functional features (and an important part of Azure IoT Hub SDK‘s device-side technical features; only a few of them have not been used ….. for now 🙂 ).

This solution main characteristics are:

  • Local  (not a web solution), fully .NET Core based.
  • Lightweight tool (not a whole platform)
  • Easy integration with developer tools
  • Could be used for unit tests
  • Solution configurable in many aspects (certain parts of message contents need to be improved to avoid limitations)
  • Containerizable (Docker)
  • Could be used for building a whole simulation platform (scaling out with Kubernetes or similar)
  • Functional and technical features (covers an important part of client capabilities). Implemented functional features cover:
    • Cloud To Device (C2D)
    • Device To Cloud (D2C)
    • IoT Hub Modules
    • IoT Hub Twin (Tags and Properties)
    • More details here.
  • Free (unless you decide to build a whole simulation platform with Kubernetes, etc)

 

NOTE

The solution could even be deployed in a real IoT device as a binary application (.NET Core based) as well as a Docker container.

 

The simulation logic remains quite basic (simple random logic). However, the solution is extendable with custom logic implementations.

Message structures are configurable and rely on JSON templates. A strong link exists between simulation services and JSON templates. This should be reviewed in upcoming versions and reduce that level of coupling.

Other development tools

We cannot forget the tools available for Visual Studio Code and Visual Studio :

  • extension for Visual Studio Code
  • natively integrated into Visual Studio through the Azure workload and the Cloud explorer

Even though their philosophy is different from Azure IoT Device Simulator or Microsoft Azure Device Simulator’s, these tools add IoT capabilities directly into the IDE.

 

Visual Studio Code - IoT features
Visual Studio Code – IoT features

 

Visual Studio 2019 - IoT features
Visual Studio 2019 – IoT features

 

Iteration after iteration, they are getting richer and richer in terms of features.

As mentioned earlier, these extensions are:

  • development-oriented
  • IDE-oriented

They can be used for development or testing.

They do not allow:

  • simulating whole IoT device flows (not all the IoT flows are implemented. They are not built for that either.)
  • building platforms of devices.
  • building message advanced logic or templating.
  • industrializing the simulations.

 

Conclusion

Azure IoT Device Simulator is a complementary tool for IoT developers or testers. It can also be used for building whole IoT device simulation platforms. By default, it includes three types of messages (telemetry, error, commissioning). Message schema relies on JSON templates and, except a few dependencies with the implementation of the simulation service, the schema is totally customizable. The solution implements an important set of Azure IoT SDK capabilities covering a large field of IoT scenarios.
Azure IoT Device Simulator can be used as a standalone tool or as part of a global platform (Docker + Kubernetes or binaries + Apps).

Microsoft Device Simulation Accelerator is less technical, IDE-oriented (web) and very fast to deploy. In a few minutes, a whole IoT Device simulation platform can be built. The types of messages and the metrics sent by devices can be customized totally (it seems that the current version does not allow customizing the simulation logic). The current version of the platform seems to be data generation-oriented. Costs also need to be taken into account; Microsoft Device Simulation Accelerator is a SaaS solution (web front).

Azure IoT Device Simulator Vs Microsoft Device Simulator (Accelerator)

Azure IoT Device Simulator

What is this?

Azure IoT Device Simulator was initially created as a developer tool for my personal IoT projects. Quite soon I felt the need of making it more reusable and generic.

Knowing that all IoT Developers are confronted with this same need, I thought it was maybe useful either build a monetizable solution (but with the required support, etc) or publish it as an opensource project. Great question because both alternatives were quite attractive to me.

In the end, I decided to publish the solution as an Open Source project and make it live through contributions. That will allow me to focus on other side projects and I am pretty sure I will have the chance to live new business adventures quite soon :).

The solution is available here.

I hope you will find it useful. Feel free to contribute and improve it.

Enjoy :).

 

Features

The Azure IoT Device Simulator has for purpose to help IoT developers and testers. Indirectly it should help business too, accelerating part of IoT projects and reducing costs.

The solution implements different types of Cloud To Device (C2D) / Device To Cloud (D2C) flows between Microsoft Azure IoT Hub and the simulated device.

More details here.

Example of uses:

  • development tool for developers working in Microsoft Azure IoT solutions (cloud)
  • tester tool in IoT-oriented projects
  • scalable IoT simulation platforms
  • fast and simple development of IoT devices

Azure IoT Device Simulator logs

Azure IoT Device Simulator Logs

Global features

Many of the functional features implemented are:

  • device simulation
  • module simulation
  • device simulation configuration (JSON-based)
  • oriented module oriented configuration (JSON-based)
  • no specific limitation on the number of modules (only limited by IoT Hub constraints)
  • containerizable (Docker image)
  • message templates based on JSON
  • implementation of full IoT flows (C2D, D2C, C2M, M2C)

Functional features

Device-level (C2D/D2C)

Commands

  • Request latency test
  • reboot device
  • device On/Off
  • read device Twin
  • generic command (with JSON payload)
  • generic command
  • update telemetry interval

Messages The device can send messages of different types (telemetry, error, commissioning).

Twin Any change in the Desired properties is handled by the device.

Module-level (C2M/M2C)

The features at the module level are identical to the device features except for the latency tests.

Details.

Technical core

Global technical features

Functional features are based on these generic technical features:

  • telemetry sent from a device.
  • a device can contain one or many modules.
  • each module behaves independently with its own flows (C2M/M2C) and its configuration settings.
  • telemetry sent from a module.
  • messages received by a device.
  • messages received by a module.
  • commands received by a device.
  • commands received by a module.
  • Twin Desired properties changed notification (for devices).
  • Twin Desired properties changed notification (for modules).

D2C

Device-level

  • IoT Messages
  • Twins (Reported)

Module-level (M2C)

  • IoT Messages
  • Twins (Reported)

C2D

Device-level

  • Twins (Desired)
  • Twins (Tags)
  • Direct Methods
  • Messages

Module-level (C2M)

  • Twins (Desired)
  • Twins (Tags)
  • Direct Methods
  • Messages

Technical background

Development based on:

  • .NET Core 2.x
  • Microsoft Azure IoT SDK (Device capabilities, including IoT Hub modules)

 

Delivery type:

 

 

Azure IoT Device Simulator