Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WEB]: Accessibility focus border doesn't follow when navigating through interactive elements with tab key #52106

Closed
838 opened this issue Mar 6, 2020 · 17 comments · Fixed by flutter/engine#25797
Assignees
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) f: focus Focus traversal, gaining or losing focus found in release: 1.22 Found to occur in 1.22 framework flutter/packages/flutter repository. See also f: labels. P1 High-priority issues at the top of the work list platform-web Web applications specifically r: fixed Issue is closed as already fixed in a newer version

Comments

@838
Copy link
Contributor

838 commented Mar 6, 2020

Screen Reader does not follow the focus when navigating through interactive elements with tab-key.

ezgif-7-e905ef69a6af

@TahaTesser TahaTesser added a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) f: focus Focus traversal, gaining or losing focus framework flutter/packages/flutter repository. See also f: labels. platform-web Web applications specifically labels Mar 6, 2020
@yjbanov yjbanov added this to the Near-term Goals milestone Mar 12, 2020
@yjbanov
Copy link
Contributor

yjbanov commented Mar 12, 2020

Could you please provide a small code snippet that reproduces this issue?

/cc @nturgut

@yjbanov yjbanov added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Mar 12, 2020
@nturgut
Copy link
Contributor

nturgut commented Mar 12, 2020

Which flutter channel are you running the app? Can you also provide a flutter doctor output?

@838
Copy link
Contributor Author

838 commented Mar 14, 2020

Hi @yjbanov and @nturgut

The same issue could also be reproduces on Gallery App

the local project a11y_web_demo.zip

flutter doctor output:

[✓] Flutter (Channel master, v1.15.10-pre.4, on Mac OS X 10.15.3 19D76, locale en-DK)
    • Flutter version 1.15.10-pre.4 at /Users/b044560/Development/flutter
    • Framework revision c2ae654ddf (9 days ago), 2020-03-05 08:03:26 +0100
    • Engine revision 7df0a6e35e
    • Dart version 2.8.0 (build 2.8.0-dev.11.0 8e5354dccb)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/b044560/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • ANDROID_HOME = /Users/b044560/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.3.1, Build version 11C504
    • CocoaPods version 1.9.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.43.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.8.1

[✓] Connected device (2 available)
    • Chrome     • chrome     • web-javascript • Google Chrome 80.0.3987.132
    • Web Server • web-server • web-javascript • Flutter Tools

@no-response no-response bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Mar 14, 2020
@nturgut
Copy link
Contributor

nturgut commented Mar 22, 2020

Thanks for filing the issue. I was able to create it. I'll have a look soon.

@nturgut nturgut self-assigned this Mar 22, 2020
@nturgut nturgut modified the milestones: Near-term Goals, April 2020 Mar 22, 2020
@mdebbar
Copy link
Contributor

mdebbar commented Apr 2, 2020

It looks like the problem is we don't inform the browser when Flutter's focus node has changed. We need to figure out a way to notify the browser and screen reader that focus has changed.

@nturgut nturgut modified the milestones: April 2020, Near-term Goals Apr 27, 2020
@nturgut nturgut removed their assignment Apr 27, 2020
@kf6gpe kf6gpe added the P1 High-priority issues at the top of the work list label May 29, 2020
@yjbanov yjbanov added P2 Important issues not at the top of the work list and removed P1 High-priority issues at the top of the work list labels Jun 16, 2020
@kf6gpe kf6gpe modified the milestone: [DEPRECATED] Near-term Goals Jul 7, 2020
@kf6gpe kf6gpe removed this from the [DEPRECATED] Near-term Goals milestone Jul 22, 2020
@Hixie Hixie removed this from the None milestone Aug 17, 2020
@TahaTesser
Copy link
Member

TahaTesser commented Sep 18, 2020

In my case, even the focus window even doesn't show, sometimes tiny box appear in random above these elements
Screenshot 2020-09-18 at 8 29 19 PM

code sample
import 'package:flutter/material.dart';
import 'package:howler/howler.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  double progressValue = 0;

  bool isPlaying = false;

  Howl howl = Howl(
    src: [
      'https://lbs.nota.dk/api/v1.0/files/e8e60b08-8983-4ea9-a3fe-0e79e020ce3a/37027/0/05_kapitel_1.mp3',
      'https://lbs.nota.dk/api/v1.0/files/b8e9dcde-f172-4047-815b-7889ba877c9f/37027/0/06_kapitel_2.mp3'
    ],
  );

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Container(
              child: Row(
                mainAxisAlignment: MainAxisAlignment.center,
                children: <Widget>[
                  Semantics(
                    label: 'Skip Back',
                    button: true,
                    child: RaisedButton(
                      child: Icon(Icons.skip_previous),
                      onPressed: () {
                        print('skipBack');
                      },
                    ),
                  ),
                  Semantics(
                    label: 'Seek Back',
                    button: true,
                    child: RaisedButton(
                      child: Icon(Icons.replay_10),
                      onPressed: () {
                        this.howl.setSeek(this.howl.getSeek() - 10);
                        print('seekBack');
                      },
                    ),
                  ),
                  Semantics(
                    label: '${this.isPlaying ? 'pause' : 'play'} button',
                    focusable: true,
                    button: true,
                    child: RaisedButton(
                      child: Icon(
                        this.isPlaying ? Icons.pause : Icons.play_arrow,
                      ),
                      onPressed: () {
                        if (!this.howl.playing()) {
                          setState(() {
                            print('play');
                            this.isPlaying = true;
                            this.howl.play();
                          });
                        } else {
                          setState(() {
                            print('pause');
                            this.isPlaying = false;
                            this.howl.pause(null);
                          });
                        }
                      },
                    ),
                  ),
                  Semantics(
                    label: 'Seek Forward',
                    button: true,
                    child: RaisedButton(
                      child: Icon(Icons.forward_10),
                      onPressed: () {
                        print('seekForward');
                        this.howl.setSeek(this.howl.getSeek() + 10);
                      },
                    ),
                  ),
                  Semantics(
                    label: 'Skip Forward',
                    button: true,
                    child: RaisedButton(
                      child: Icon(Icons.skip_next),
                      onPressed: () {
                        print('skipNext');
                      },
                    ),
                  ),
                  Semantics(
                    label: 'Progress bar',
                    child: Slider(
                      value: progressValue,
                      onChanged: (value) {
                        setState(() {
                          progressValue = value;
                        });
                      },
                    ),
                  )
                ],
              ),
            ),
          ],
        ),
      ),
    );
  }
}
flutter doctor -v
[✓] Flutter (Channel master, 1.22.0-10.0.pre.251, on Mac OS X 10.15.6 19G2021
    x86_64, locale en-GB)
    • Flutter version 1.22.0-10.0.pre.251 at
      /Users/tahatesser/Code/flutter_master
    • Framework revision d26268bb9e (13 hours ago), 2020-09-18 09:06:26 +0900
    • Engine revision 2abe69c608
    • Dart version 2.10.0 (build 2.10.0-136.0.dev)

 
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/tahatesser/Code/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/tahatesser/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0, Build version 12A7209
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 49.0.2
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.49.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.14.1

[✓] Connected device (3 available)
    • macOS (desktop)  • macos      • darwin-x64     • Mac OS X 10.15.6 19G2021
      x86_64
    • Web Server (web) • web-server • web-javascript • Flutter Tools
    • Chrome (web)     • chrome     • web-javascript • Google Chrome
      85.0.4183.102

• No issues found!

@TahaTesser TahaTesser added found in release: 1.22 Found to occur in 1.22 has reproducible steps The issue has been confirmed reproducible and is ready to work on labels Sep 18, 2020
@TahaTesser TahaTesser changed the title [WEB]: Accessibility VoiceOver border Focus doesn't follow when navigating through interactive elements with tab key [WEB]: Accessibility focus border doesn't follow when navigating through interactive elements with tab key Sep 21, 2020
@nturgut
Copy link
Contributor

nturgut commented Sep 21, 2020

@838 I have a question to assess the priority of the issue.

When using screen readers, we navigate using the screen reader shortcuts. Why is it necessary for tab key to be used when screen reader is on? Is it common usage for tab key to be used instead of "VoiceOver Control key+left right arrows keys"?

@838
Copy link
Contributor Author

838 commented Sep 21, 2020

@nturgut

The main reason to navigate by the tab key is to reduce the navigation steps by only navigating through the interactive elements like links and buttons instead of navigating through every element on the page.

As an example, let's say we have a page with a list of books and each book with data(title, author, last played, etc) and a play button. Usually, the tab key is used to only jump through the play buttons.

Although every ScreanReaders have their own nice features for faster navigation, still the tab key is very common and widely used especially for users who are new and are not aware of the power of the ScreenReaders.

@nturgut
Copy link
Contributor

nturgut commented Sep 21, 2020

@838 thanks for the comments.

@mariamhas Heads up, looks like this use case (using tab key when screen-reader is on) is not working.

@nturgut nturgut self-assigned this Dec 11, 2020
@nturgut nturgut added P1 High-priority issues at the top of the work list and removed P2 Important issues not at the top of the work list labels Dec 11, 2020
@nturgut
Copy link
Contributor

nturgut commented Dec 11, 2020

@mariamhas

Looks like this issue is not a blocker for Narrotor, therefore I'm re-reducing the priority back to P4.

@nturgut nturgut removed their assignment Dec 11, 2020
@nturgut nturgut added P2 Important issues not at the top of the work list and removed P1 High-priority issues at the top of the work list labels Dec 11, 2020
@nturgut
Copy link
Contributor

nturgut commented Dec 11, 2020

Comment so far, thanks to @mdebbar suggestion, I tried utilizing the focus flag received with the semantics updates. As expected focus correctly follows the tab key. However when we want to change the focus as we do with the input fields, the screen reader does not focus on the next element.

semanticsElement.focus();

I also tested it with a simple jsfiddle page. Looks like screenreader does not focus on an item such as div role="checkbox" aria-checked="false"> C1 </div> It only focusses on buttons and input fields. One way we can track this issue is using input elements (as we do for text editing) and buttons for different semantic roles (checkbox, radio, switch, button). However this will be a bigger change for Flutter semantics.

@yjbanov @mdebbar what do you think?

@yjbanov yjbanov added P1 High-priority issues at the top of the work list and removed P2 Important issues not at the top of the work list labels Apr 2, 2021
@yjbanov yjbanov self-assigned this Apr 2, 2021
@yjbanov
Copy link
Contributor

yjbanov commented Apr 2, 2021

I'm looking into this now. This affects NVDA (and reportedly JAWS too, although I don't have JAWS yet to check).

@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 31, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) f: focus Focus traversal, gaining or losing focus found in release: 1.22 Found to occur in 1.22 framework flutter/packages/flutter repository. See also f: labels. P1 High-priority issues at the top of the work list platform-web Web applications specifically r: fixed Issue is closed as already fixed in a newer version
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants