Robot Framework Selenium - Cannot reliably click/scroll in Flutter web canvas elements

Question

I'm internship QA testing on Flutter web application with Robot Framework and Selenium Library. My task is to perform end-to-end (E2E) testing on a Flutter web app that heavily uses canvas structure. The test cases involve basic interactions such as clicking, pointing, focusing, and scrolling to elements. Initially, my team suggested using aria-label attributes as keys for identifying elements, which worked well for basic actions.

However, I've encountered several issues when trying a test scenario involving:

Logging in

Selecting a menu item

Editing profile data

Scrolling within a card

Returning to the homepage

The problem starts from the login step, where using standard Selenium actions (Click Element and Input Text) results in errors saying the elements are intercepted or hidden by Flutter. To bypass this, I used the Press Keys function to simulate typing and pressing the Enter key, which allowed me to log in.

After logging in, I was able to interact with menu elements using key presses, but the next major issue arose with scrolling. The website has a scrollable card component, and none of the methods I've tried (Scroll Element Into View, document.scrollIntoView(), or window.scroll) seem to fully scroll the element into view. They only scroll partially or don't scroll at all, leaving elements out of view.

Here's my some basic example Robot Framework test:

Noted That I've used multiple Sleep commands because the web application loads slowly, and these pauses ensure elements have enough time to load properly.

Test Business Profile
Open Browser    ${URL}  ${BROWSER}    
Sleep   5s
Maximize Browser Window
Wait Until Element Is Visible   ${login_email}
Press Keys    ${login_email}    ${Email}
Sleep   1s
Wait Until Element Is Visible   ${login_pass}
Press Keys    ${login_pass}   ${KEY_ENTER}
Sleep   1s
Wait Until Element Is Visible   ${login_button}
Press Keys  ${login_button}    ENTER
Sleep   1s
Close Browser
Press Keys   ${organization_management}  ENTER
Sleep   1s
Press Keys   ${business_unit_management}    ENTER
Sleep   1s
Press Keys   ${add_business_unit}  ENTER
Sleep   1s
Press Keys   ${buname_field}  TESTING NAME FIELD
Sleep   1s
Press Keys   ${budescription_field}  TESTING DESCRIPTION FIELD
Sleep   1s
Scroll Element Into View    ${submit_button}
Sleep   1s
Press Keys   ${submit_button}  ENTER
Sleep   3s
Press Keys   ${confirm_button}  ENTER
Sleep   3s
Close Browser

Could someone experienced in Robot Framework and Selenium testing with Flutter web apps provide guidance or recommend workarounds to handle interactions reliably, especially control scrolling and focus Element within Flutter canvas structures?

of course, I'm trying asking ChatGPT ,Sonnet and OTHER AI b4 posting

Some error picture enter image description here enter image description here

dbc · Accepted Answer · 2025-05-12 15:43:23Z

I share your frustration getting scrolling working within a flutter web app.. Here is a solution I did get to work:

Scroll Container Until Element Visible
    [Arguments]    ${container_xpath}    ${target_xpath}    ${max_attempts}=20    ${scroll_by}=100
    Execute Javascript    document.evaluate("${container_xpath}",document,null,XPathResult.FIRST_ORDERED_NODE_TYPE,null).singleNodeValue.focus();

    FOR    ${index}    IN RANGE   ${max_attempts}
        ${visible}=    Run Keyword And Return Status    Element Should Be Visible    ${target_xpath}    timeout=0.1s
        Exit For Loop If    ${visible}
        Execute Javascript    document.evaluate("${container_xpath}",document,null,XPathResult.FIRST_ORDERED_NODE_TYPE,null).singleNodeValue.scrollBy(0,${scroll_by});
        Sleep    0.2s
    END

The trick for me was getting the scrolling container to have focus, then using javascript scrollBy.. Otherwise it would not scroll to where I needed it to..

I then call the Keyword with this:

Scroll Container Until Element Visible  //flt-semantics[@role='group' and contains(@style,'overflow-y')]   //*[contains(text(),"Acknowledge Reading Declaration")]/parent::*/following::flt-semantics[1]    20    -100

So the container element identifier followed by the element you want visible on the page.

Collectives™ on Stack Overflow

Robot Framework Selenium - Cannot reliably click/scroll in Flutter web canvas elements

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related