I am developing a mod for a game with a Lua api.

I am trying to detect if I am looking through from behind a window when I shoot an enemy. For context, when I shoot through this window I apply a special buff effect.

Characters lie on a map which is an xy coordinate system with 4 quadrants. 0, 0 lies at the center of the map.

I am doing a top-down view. I can get the xy position of my character, the xy position of the enemy character, and the 2 xy positions of the boundaries of the window.

I got the angle of the character relative to the angle of the window edges, and I got the angle of the character relative to the angle of the enemy.

here is an example:

player position -149.55355834961 -54.274295806885
target position -147.99998474121 -67.999992370605
left position -151.8394317627 -56.843517303467
right position -147.86442565918 -56.397045135498
angle target 96.457664489746
angle left 48.340045928955
angle right 128.51026916504

this is how I am calculating the angles (the game uses xz as the horizontal plane):

function getAngleBetween(object1, object2)
    -- y is height here
    local x1, y1, z1 = GetWorldPosition(object1)
    local x2, y2, z2 = GetWorldPosition(object2)
    local zDiff = z1 - z2
    local xDiff = x1 - x2
    local phi = math.atan2(zDiff, xDiff)
    if phi < 0 then
        phi = phi + (2 * math.pi)
    end
    local degrees = phi * (180/math.pi)
    return degrees
end

I used this code to check if the angle with the enemy was between the side of the window, but it gives inconsistent results. I think it has to do with how I am calculating the angles, and the coordinate system. For example, If I am facing north, it works. But if I am facing south, it doesn't.

    function isAngleBetweenDeg(leftAngle, rightAngle, checkAngle)
        
        leftAngle = math.mod(leftAngle,360)
        rightAngle =  math.mod(rightAngle,360)
        checkAngle =  math.mod(checkAngle,360)

        if leftAngle > rightAngle then
            return checkAngle >= leftAngle or checkAngle <= rightAngle
        end
        return checkAngle > leftAngle and checkAngle < rightAngle
    end

How do I check if I am behind the window? as far as the game's API, you only need to know GetWorldPosition give me the coordinates.