Remote Debugging .NET Containers
Vous préférez lire en français? Consulter la version française ici : Déboguer les conteneurs .NET à distance
You will find several articles that will talk about local debugging .NET container through the magic of Visual Studio or VS Code, but it’s not easy to find the right information for remote debugging.
Local debugging with the different IDEs is much easier because they install the .NET debugger in the container for us without telling us and configure it properly. Compared to traditional remote debugging for .NET Framework, debugging for .NET Core does not require the use of msvsmon.exe. Instead, you need to use vsdbg provided by OmniSharp.
Local debugging requires us to run our container on our machine from the compiled source code. This is convenient during the development phase when we produce our code, but it is less convenient when we need to fix an anomaly that occurs in a subsequent phase of our delivery process. In this case, local debugging of a container can be long and tedious given the time required to retrieve the correct version of the image to be executed and its dependencies.
In this article, we will see how to debug a remote Linux Container that contains an ASP.NET Core application. For reference, here is a Dockerfile generated by Visual Studio for an ASP.NET Core application:
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging. FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base WORKDIR /app EXPOSE 80 EXPOSE 443 FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build WORKDIR /src COPY ["TestRemoteDebugK8s.csproj", "/src"] RUN dotnet restore "TestRemoteDebugK8s.csproj" COPY . . WORKDIR "/src" RUN dotnet build "TestRemoteDebugK8s.csproj" -c Release -o /app/build FROM build AS publish RUN dotnet publish "TestRemoteDebugK8s.csproj" -c Release -o /app/publish FROM base AS final WORKDIR /app COPY --from=publish /app/publish . ENTRYPOINT ["dotnet", "TestRemoteDebugK8s.dll"]
Build once, deploy everywhere
A simple search will show you several debugging examples that require you to modify the Dockerfile. The simplest approach is to include the .NET Core vsdbg debugger directly in our container image. It is simple to understand, but it complicates the developer’s job because he has to generate a new docker image and redeploy it before he can debug.
In DevOps, we favor the Build once, deploy everywhere principle. The goal here is to minimize the chances of introducing anomalies in our application by recompiling after testing. If all the tests of our application are done on a Debug version that potentially includes the debugger and that must be recompiled in Release before delivering to production, we introduce a risk of failure or we have to redo more tests. This is why we try as much as possible to deliver the binary that was used during the testing phase and, in this case, the image of the container used for the tests.
The approach of modifying the Dockerfile therefore takes us away from our Build once, deploy everywhere approach. Instead, it is possible to use the Release version of our application/image for the first tests, as this configuration includes the debugging symbols by default.
Installing the debugger
The first thing we need to do to be able to debug our application is to install the debugger. As mentioned before, it is not included in the basic container image. This allows us to keep the image size as small as possible. When we debug a Container locally, the IDE associates a volume to our container that contains the .NET Core debugger. When debugging remotely, since the container image is already created, it is preferable to install it manually.
IMPORTANT: Depending on your container base image, the following script may need to be modified. Getting packages under Linux varies from one distribution to another. For the example, I used the aspnet:3.1-buster-slim base image which is based on Debian.
To install the debugger, we must first connect in interactive mode inside the container :
Docker:
docker exec -it remote /bin/bash
Here, remote is the name given to the container at startup.
Kubernetes :
kubectl exec -it remote-7ff879f496-5jvdg -- /bin/bash
Here, remote-7ff879f496-5jvdg is the name of the pod containing the container to be debugged.
Once connected, you need to run the following script:
apt update && \ apt install -y unzip procps && \ curl -sSL https://aka.ms/getvsdbgsh | /bin/sh /dev/stdin -v latest -l ~/vsdbg
Note the following:
- The first line allows the apt manager package to update its local list of packages without installing them.
- The second line installs unzip (required by the getvsdbgsh script provided by Microsoft) and procps (required by the IDEs to list the processes running in the container).
- The last line fetches the debugger installation script and installs it to the /root/vsdbg location inside the container.
You will have noticed that you need to access the container in interactive mode to install what you need.
Attaching to the process
We now have a .NET Core application in a container with a debugger installed right next to it. Since you can’t just press F5 to start debugging, you have to attach to the process running inside the container. A valid approach would be to open an SSH channel to the container, but this is something that can be simplified by using the features of our favorite IDE.
With VS Code, you need to create a debugging launch configuration specific to the runtime context:
Docker
When our container simply runs under Docker, the following configuration can be used:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickProcess}"
},
{
"name": ".NET Core Docker Attach",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickRemoteProcess}",
"pipeTransport": {
"pipeProgram": "docker",
"pipeArgs": [ "exec", "-i", "remote" ],
"debuggerPath": "/root/vsdbg/vsdbg",
"pipeCwd": "${workspaceRoot}",
"quoteArgs": false
},
"justMyCode":false,
"sourceFileMap": {
"/src/": "${workspaceRoot}"
}
}
]
}
Launch.json file under .vscode folder
The interesting part is highlighted. Note the following:
${command:pickRemoteProcess}:
Tells VS Code to open a pick list of the process to be debugged in the container. This is why procps had to be installed previously.
“pipeProgram”: “docker” and “pipeArgs”: [ “exec”, “-i”, “remote” ], :
Tells VS Code the command line to execute when starting the debugger.
“debuggerPath”: “/root/vsdbg/vsdbg”, :
Indicates where to find the debugger in the container file system. This path is linked to the end of the line seen above :
curl -sSL https://aka.ms/getvsdbgsh | /bin/sh /dev/stdin -v latest -l ~/vsdbg
“sourceFileMap”: {
“/src/”: “${workspaceRoot}”:
This parameter tells the debugger where to find the source files for debugging. Since the default Dockerfile template copies the source files to a src directory in the container, we must indicate that the src directory actually corresponds to our working directory in VS Code.
Breakpoints do not work
If you have copied the configuration mentioned above entirely, you probably won’t have any problems with breakpoints. It is important to mention that, since we are using a Release version of our .NET Core application, the debugger will not be able to load the symbol (pdb) files automatically. You just need to specify the parameter “justMyCode”:false for the magic to work. By doing so, breakpoints will work normally.
Kubernetes
For Kubernetes, it is recommended to use the Kubernetes extension for VS Code. It makes it much easier to connect to a pod to debug it. However, you will still have to install the debugger yourself beforehand. Once this is done, you can use the context menu of a pod from the Kubernetes extension as follows :
However, the default configuration sets the justMyCode parameter to true. If you want to stay in the same workflow as mentioned at the beginning of the article (Build once, deploy anywhere), you must be able to set this parameter to false. The extension does not yet allow to change the configuration used to attach the debugger. So we have to use our own debug configuration as follows :
{
"name": ".NET Core K8s Attach",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickRemoteProcess}",
"pipeTransport": {
"pipeProgram": "kubectl",
"pipeArgs": [ "exec", "-i", "remote-75c859fc4c-gcx7d", "--" ],
"debuggerPath": "/vsdbg/vsdbg",
"pipeCwd": "${workspaceRoot}",
"quoteArgs": false
},
"sourceFileMap": {
"/src/TestRemoteDebugK8s/": "${workspaceRoot}"
},
"justMyCode":false
}
Launch.json file under the .vscode directoryNote that the name of the pod to be debugged will have to be modified according to your context.
Conclusion
I hope I’ve been able to clearly explain how to remotely debug a .NET Core container. Your environment may be a bit different if you have changed your base image or if your privileges are restricted in the runtime environment. You can always use the above examples to adapt them to your context.
A good application security practice is to use base images that contain only the bare essentials. The distroless images published by Google have been worked to eliminate what is not necessary to run your application. However, these images are problematic for debugging, as they do not contain a Shell that allows tools to be installed after the image is created. In Kubernetes, the use of Ephemeral Containers may help us with the arrival of Kubernetes version 1.18. A good subject for a next article!
Recent Comments