Issues with ASPNET Identity Google behind NGINX based reverse proxy

I'm in the process of adding Google Authentication to .NET 8 web app behind NGINX reverse proxy.

App itself is simple CRUD web app. I do have Let's Encrypt certificate for https, it is terminating on proxy and traffic between app and proxy is plain http. I am aware that X-Forwarded-* headers needs to be set on reverse proxy, it is configured, and up until now app seemed to function normaly - it does return some direct links to images/etc. and those are returned as https://*, so I'm quite confident that everything works as it should.

However, recently I decided to add Google Auth to it and started experiencing some issues I've been banging my head at for couple of days now.

For implementation I'm using provided MS Identity libraries and I think something unexpected is happening in one of them, but it is quite hard to catch the issue.

Google side seems to be configured OK, I have client Id and Secret, added my callbacks to allow list (https://www.example.com/signin-google). When doing login from application side - initial call to google comes through, user gets to choose his identity (if multiple) and it looks like initial callback (to /signin-google, which is handled by Identity libraries) comes back OK, however problem happens after that. As far as I understand, Identity library does a http(s) call to get user info to google after /signin-google callback is received and it throws an exception. It looks like this user info request gets rejected by google with a reason "Bad Request", redirect_uri_mismatch. Not confirmed (have no idea how to do that), but I suspect that somehow Identity library is not able to identity that it is behind reverse proxy and sends some callback uri as plain http. Just to reiterate - this is the only case in whole application when I see this effect that application is not understanding that it is behind https terminating proxy.

While doing just plain http and with Google in development mode (production does not allow http, only https) everything works as expected.

Did sample app based on VS template and seeing same issue - initial login screen shows up ok, however exception thrown inside callback handler:

var builder = WebApplication.CreateBuilder(args);

var connectionString = builder.Configuration.GetConnectionString("DefaultConnection") ?? throw new InvalidOperationException("Connection string 'DefaultConnection' not found.");
builder.Services.AddDbContext<ApplicationDbContext>(options => options.UseSqlServer(connectionString));
builder.Services.AddDatabaseDeveloperPageExceptionFilter();
builder.Services.AddDefaultIdentity<IdentityUser>(options => options.SignIn.RequireConfirmedAccount = true).AddEntityFrameworkStores<ApplicationDbContext>();

builder.Services
    .AddAuthentication()
    .AddGoogle(options =>
    {
        options.ClientId = "deleted";
        options.ClientSecret = "deleted";
    });

//accepting everything for testing purpouses
builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders = ForwardedHeaders.All;
    options.ForwardLimit = null;

    options.KnownProxies.Clear();
    options.KnownNetworks.Clear();
});

var app = builder.Build();

app.UseForwardedHeaders();

//works as expected, I can see google login screen
app.MapGet("login", () => {
    var authenticationProperties = new AuthenticationProperties
    {
        RedirectUri = "ok"
    };

    return Results.Challenge(authenticationProperties, [GoogleDefaults.AuthenticationScheme]);
});

//never hit, receive .NET exception after i get redirected back to signin-google on app side
//the exception 100% comes from application side, just to rule out other layers - I can see "Unhandled exception" here
//when added to google options Events.OnRemoteFailure gets called, however I failed to get something usefull out of it except of already known "Bad Request" and redirect_uri_mismatch
app.MapGet("ok", () => "ok");

app.Run();

Exception page

NGINX headers set on location / (tried different combos with no result):

proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header   X-Forwarded-Proto $scheme;

Looks like headers are set. I can also see "Host" is set to proper one

Both screenshots are taken from the same page, the address bar shows https://www.example.com/sigin-google?state=loadsofparamsfromgoogle

Tried experimenting with reverse proxy settings, but I think all headers are set correctly. Tried experimenting with app, changing known proxies, used headers - nothing helped.

At this moment I'm not even sure where lies the problem, because application (and login flow) partially works...

Update with more experiments:

Wrote even simpler app to rule out as many variables as possible, but for me it still looks like an issue not on proxy side...

var builder = WebApplication.CreateBuilder(args);

builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders = ForwardedHeaders.All;
    options.ForwardLimit = null;

    options.KnownProxies.Clear();
    options.KnownNetworks.Clear();
});

var app = builder.Build();

app.Use((ctx, next) => { Console.WriteLine(ctx.Request.IsHttps); return next(ctx); });

app.UseForwardedHeaders();

app.Use((ctx, next) => { Console.WriteLine(ctx.Request.IsHttps); return next(ctx); });

app.MapGet("/", () => "Works!");

app.Run();

The outputs are:

http://localhost - False, False

https://localhost - True, True

https://example.com (hosted behind TLS terminating proxy) - False, True

This leads me to believe that X-Forwarded headers information is getting applied, but is somehow not used by Identity Google library? Notice how Request.IsHttps switches to True after UseForwardedHeaders()