onAccessibilityEvent Action_click Not working

Question

Okay, so I am working on an app that will auto accept lyft request, but I am having a problem with my code not using performAction(AccessibilityNodeInfo.ACTION_CLICK); correctly.

public class AutoService extends AccessibilityService {
private static LyftAdapter lyftAdapter = new LyftAdapter();

// Automated Service (onAccessibilityEvent)
@TargetApi(16)
@Override
public void onAccessibilityEvent(AccessibilityEvent event)
{
    AccessibilityNodeInfo source = event.getSource();
    String lyftPackage = "com.lyft.android.driver";
    String packageName = Tools.getPackage(source);

    if (!packageName.equals(lyftPackage))
    {
        event.recycle();
        return;
    }

    if (source == null)
    {
        event.recycle();
        return;
    }

    processUI(event.getSource());
}

public void processUI(AccessibilityNodeInfo source)
{
    source = getRootInActiveWindow();

    if (Tools.getPackage(source).equals("com.lyft.android.driver") || Tools.getPackage(source).equals("me.lyft.android"))
    {
        if (!Lyft_Status.equals("OFFLINE"))
        {
            lyftAdapter.processEvent(source);
        }
        else
        {
            Log.v(TAG, "Can't process UI: " + Lyft_Status);
        }
    }

    if (source != null)
        source.recycle();
}
}

public abstract class RideshareAdapter {
public void processEvent(final AccessibilityNodeInfo source)
{
        final StringBuilder sb = new StringBuilder();
        processSubEvent(source, 0, sb);
        final String string = sb.toString();

        if (string == null)
        {
            Log.v(TAG, "String is NULL");
            return;
        }

        processUIText(source, string.toLowerCase());
}

// PROCESS SECONDARY EVENT
private void processSubEvent(final AccessibilityNodeInfo source, final int n, final StringBuilder sb) {
    for (int i = 0; i < n; ++i) {
        sb.append("\t");
    }

    if (source != null)
    {
        sb.append(Tools.getText(source));
        sb.append("\n");
        final int childCount = source.getChildCount();

        for (int j = 0; j < childCount; ++j) {
            final AccessibilityNodeInfo child = source.getChild(j);
            processSubEvent(child, n + 1, sb);
            if (child != null) {
                child.recycle();
            }
        }
    }
}

// CLICK THE SCREEN
protected void clickScreen(AccessibilityNodeInfo source, final String text)
{
    final AccessibilityNodeInfo s = source;

    new Handler().postDelayed(new Runnable() {
        List<AccessibilityNodeInfo> list = s.findAccessibilityNodeInfosByText(text);
        @Override
        public void run() {
            for (final AccessibilityNodeInfo node : list) {
                node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
        }
    }, 1000);
}
}

public class LyftAdapter
    extends RideshareAdapter
{
// LYFT ADAPTER
protected void processUIText(AccessibilityNodeInfo source, String text)
{        
    // RIDE REQUEST
    if (text.contains("tap here to accept"))
    {
        clickScreen(source, "Tap here to accept");
    {  
}         

The string comes out as (Just like it is shown):
Lyft
11 mins
away
Passenger Name
New
Tap here to accept

But for some reason, it triggers saying it is going to click on "Tap here to accept" textview, but it never actually does it. Any suggestions?

It could be a number of things. First of all, add a breakpoint and check that execution is reaching the line node.performAction(AccessibilityNodeInfo.ACTION_CLICK); . Then make sure the node is actually clickable. Finally, but not less important, DO ALL THE WORK IN THE MAIN THREAD. If you use Handler.postDelayed you are creating a new thread, that will execute a second latter, and the AccessibilityEvent may be useless by then. — Abraham Toledo
– Abraham Toledo, Commented Jul 31, 2017 at 4:36
By doing all the work in the main thread you garantee that the system will wait for you to finish before destroying the AccessibilityEvent. — Abraham Toledo
– Abraham Toledo, Commented Jul 31, 2017 at 4:37
Problems from the delay have nothing to do with the event itself. Just the nodes and screen content. If the screen hasn't changed, the nodes in the event would still be valid. The delay is DEFINITELY a bad idea, but you got the reasoning wrong, and it's a very important difference. The event will be fine, there is no guarantee however on the screen contents. — MobA11y
– MobA11y, Commented Jul 31, 2017 at 5:52
Well the delay was put in, so that it wouldn't click the same textview twice in a row. — Jayce
– Jayce, Commented Jul 31, 2017 at 12:42

MobA11y · Accepted Answer · 2017-07-31 13:48:29Z

2

To be completely honest, your post is very difficult to read. You have functions that you have defined purely for organizational purposes and not because they are meant to be re-used. It makes it very difficult to parse and understand over the course of a StackOverflow post... Yet you did not provide enough for me to copy and paste and make sense of in Android Studio.

When you post code on StackOverflow you should go for a minimal replicating example and you ABSOLUTELY should remove your random Log calls. You may need them to help you understand what's happening, but hopefully WE do not :) and they just clutter things and make it more difficult to read your code. THIS BEING SAID, allow me to focus on one bit,

Note that I have cleaned up some of the poor style and debugging statements. Answers are in the code comments!

protected void clickScreen(final AccessibilityNodeInfo source, final String text)
{
new Handler().postDelayed(new Runnable() {
    //Find ALL of the nodes that match the "text" argument.
    List<AccessibilityNodeInfo> list = source.findAccessibilityNodeInfosByText(text);
    @Override
    public void run() {
        //Non discrliminintly click them, whether they're buttons, or text fields or links... just click them and hope they do something.
        for (final AccessibilityNodeInfo node : list) {
            node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
        }
    }
//Delay it for a second AFTER the function has been called for no particularly good reason besides perhaps invalidating all of the nodes in the heirarchy... GOOD CALL!
}, 1000);
}

Given the above issues and the aforementioned generic code quality issues, it is difficult to provide a concise answer. This post leaves too many potential issues. Any provided answer would be a stab in the dark. I find it MOST likely that the problem is covered in my code comments, but it could most definitely be elsewhere. Also, my apologies for the sass!

All this being said, you might try this version of the function!

static void clickFirstMatchingNode(AccessibilityService service, final String text) {
    final List<AccessibilityNodeInfo> list = service.getRootInActiveWindow().findAccessibilityNodeInfosByText(text);

    for (AccessibilityNodeInfo node : list) {

       //Check if the action completely successfully. Also, only click one of them. This is kind of an assumption, it also simplifies the logic. You can certainly write a version of this that clicks everything that matches!
        if (node.performAction(AccessibilityNodeInfo.ACTION_CLICK)) return;

    }

    //If no node is successfully clicked Log some stuff!
    Log.wtf(YourService.class.getName(), "Failed to click any nodes! WTF?: " + text);
}

NOTE: None of the above mentioned anything to do with your use of Accessibility APIs! I think that that is interesting.

edited Jul 31, 2017 at 13:48

answered Jul 31, 2017 at 5:31

MobA11y

19k3 gold badges51 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Jayce Over a year ago

Sorry for the difficult read, I am new to programming and this will be my first actual full app. I have updated my code to include what it is extending on each class. I can't use your suggested change of getRootInActiveWindow() unless I extend AccessibilityService into each of the separate classes.

MobA11y Over a year ago

Yes you can. Just make it a static method and take a service as an argument. I've updated.

MobA11y Over a year ago

Also, I changed the name to be more descriptive of the actual purpose and removed the delay, seeing some of your comments on it, I'm convinced it's absolutely unnecessary.

Jayce Over a year ago

ChrisCM, I have changed my code exactly like you have, but it is still clicking on the same node multiple times without the delay in there. So Lyft will go online, offline, online when my app detects the event.

MobA11y Over a year ago

Are you sure it's not responding to multiple events?

|

Collectives™ on Stack Overflow

onAccessibilityEvent Action_click Not working

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related