ASP.NET Core 7 Web API integration tests - WithHostBuilder breaking database connection

I'm working on an ASP.NET Core 7 Web API, and I'm writing integration tests using CustomWebApplicationFactory.

My program is making some external HTTP calls, so I want to mock them in my tests. I want to have granular control on what I mock and to be able to setup HTTP return values from each test.

That's why I'm using WithWebHostBuilder and then ConfigureTestServices to mock the HttpClient.

My problem is that the simple addition of the factory.WithHostBuilder call, the tests start to fail, as if the database doesn't persist anything.

I'll show you a simplified example that mimicks my approach, and in which the issue I describe persists.

My simple custom web application factory, replacing DbContext:

public class CustomWebApplicationFactory<TProgram> : WebApplicationFactory<TProgram> where TProgram : class
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureServices(services =>
        {
            ServiceDescriptor? descriptor = services.SingleOrDefault(
                d => d.ServiceType == typeof(DbContextOptions<SetsContext>));

            if (descriptor is not null)
            {
                services.Remove(descriptor);
            }

            ServiceDescriptor? dbConnectionDescriptor = services.SingleOrDefault(
                d => d.ServiceType ==
                     typeof(DbConnection));

            if (dbConnectionDescriptor != null)
            {
                services.Remove(dbConnectionDescriptor);
            }

            // Create open SqliteConnection so EF won't automatically close it.
            services.AddSingleton<DbConnection>(container =>
            {
                var connection = new SqliteConnection("DataSource=:memory:");
                connection.Open();

                return connection;
            });

            services.AddDbContext<SetsContext>((container, options) =>
            {
                DbConnection connection = container.GetRequiredService<DbConnection>();
                options.UseSqlite(connection);
            });
        });

        builder.UseEnvironment("test");
    }
}

My tests then use this factory:

public class SetsTests : IClassFixture<CustomWebApplicationFactory<Program>>
{
    private readonly CustomWebApplicationFactory<Program> _factory;

    public SetsTests(CustomWebApplicationFactory<Program> factory)
    {
        _factory = factory;
    }

    [Fact]
    public async Task Get_WithTwoSets_ReturnsJsonWithThem()
    {
        HttpClient client = _factory.CreateClient();

        await using (AsyncServiceScope scope = _factory.Services.CreateAsyncScope())
        {
            ISetsRepository setsRepository = scope.ServiceProvider.GetRequiredService<ISetsRepository>();
            SetDto set1 = await setsRepository.CreateSetAsync("123");
            SetDto set2 = await setsRepository.CreateSetAsync("456");
        }

        HttpResponseMessage response = await client.GetAsync("api/sets/");

        Assert.Equal(HttpStatusCode.OK, response.StatusCode);
        var respType = response.Content.Headers.ContentType?.MediaType;
        Assert.Equal("application/json", respType);
        SetDto[]? content = await GetResponseContentAsync<SetDto[]>(response);
        Assert.NotNull(content);
        Assert.Equal(2, content.Length);
    }
}

This simple test passes - I create 2 sets and then they are returned from the DB properly.

But just with one simple change (to inject mock services), the tests now fails on Assert.Equal(2, content.Length); check - returned content has 0 records.

// code breaks if I replace
// HttpClient client = _factory.CreateClient();
// with
HttpClient client = _factory.WithWebHostBuilder(builder => { }).CreateClient();

I have no idea why it's causing that.

My Program.cs in my API is very simple in this example:

using BrickFolio.Data;

using Microsoft.EntityFrameworkCore;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

builder.Services.AddEndpointsApiExplorer();

builder.Services.AddDataServices(builder.Configuration, builder.Environment);

WebApplication app = builder.Build();

if (builder.Environment.IsProduction() is false)
{
    using IServiceScope scope = app.Services.CreateScope();
    IServiceProvider services = scope.ServiceProvider;

    try
    {
        SetsContext context = services.GetRequiredService<SetsContext>();
        context.Database.Migrate();
    }
    catch (Exception ex)
    {
        ILogger logger = services.GetRequiredService<ILogger>();
        logger.LogError(ex, "An error occurred while migrating the database.");
    }
}

app.MapControllers();

app.Run();

public partial class Program
{
}

It only sets up controllers and calls AddsDataServices from the data project:

public static IServiceCollection AddDataServices(this IServiceCollection services, IConfiguration config, IHostEnvironment env)
{
    DatabaseOptions? databaseOptions = config.GetRequiredSection(DatabaseOptions.SectionName)
            .Get<DatabaseOptions>();

    if (databaseOptions is null)
    {
        throw new ArgumentException("DatabaseOptions not found in configuration");
    }

    var dbPath = Path.Combine(env.ContentRootPath, databaseOptions.FileName);

    services.AddDbContext<SetsContext>(options =>
            options.UseSqlite($"Data Source={dbPath}"));

    services.AddScoped<ISetsRepository, SetsRepository>();

    return services;
}

That adds DbContext.

Am I understanding WithHostBuilder wrongly? Why would it result in this unexpected behaviour?

I have tried not using the repository and just using contexts in my tests, but the result was the same.

I've also tried debugging it a bit more, and using SQLite with files instead of the memory option. I've assigned an unique Guid to the test db in each test:

        services.AddSingleton<DbConnection>(container =>
        {
            var dbFileName = $"test-{Guid.NewGuid()}.db";
            var connection = new SqliteConnection($"Data Source={dbFileName}");
            connection.Open();

            return connection;
        });

As a result, I know that without calling WithHostBuilder, it only creates one test file that has the records.

When I call WithHostBuilder, I end up with two files, one with records and the second one empty. I suspect that the same happens when I use memory Data Source - I end up with two "in memory" database connections, with tests using one connection, and the API Program using the other.

I'm only unsure how to address that.

EDIT: I realised I haven't shown my controller:

[ApiController]
[Route("api/sets")]
public class SetsController : ControllerBase
{
    private readonly ISetsRepository _setsRepository;

    public SetsController(ISetsRepository setsRepository)
    {
        _setsRepository = setsRepository;
    }

    // GET api/sets
    [HttpGet]
    [ProducesResponseType<SetDto>(StatusCodes.Status200OK)]
    [Produces("application/json")]
    public async Task<IList<SetDto>> Index()
    {
        IList<SetDto> sets = await _setsRepository.GetAllSetsAsync();
        return sets;
    }
}