inconsistencies in docs builds (Sphinx 2)

[jaraco]

BPO	36853
Nosy	@jaraco, @drj11, @Carreau, @asottile, @JulienPalard, @tirkarthi
PRs	bpo-36853: Fix suspicious.py to actually print the unused rules #13579 [3.8] bpo-36853: Fix suspicious.py to actually print the unused rules (GH-13579) #15649 [3.7] bpo-36853: Fix suspicious.py to actually print the unused rules (#13579) #15653

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-05-08.16:03:28.240>
labels = ['build', '3.8', 'docs']
title = 'inconsistencies in docs builds (Sphinx 2)'
updated_at = <Date 2019-09-02.19:17:28.460>
user = '/~https://github.com/jaraco'

bugs.python.org fields:

activity = <Date 2019-09-02.19:17:28.460>
actor = 'jaraco'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2019-05-08.16:03:28.240>
creator = 'jaraco'
dependencies = []
files = []
hgrepos = []
issue_num = 36853
keywords = ['patch']
message_count = 9.0
messages = ['341897', '341995', '341997', '343574', '343578', '345221', '351016', '351019', '351036']
nosy_count = 8.0
nosy_names = ['jaraco', 'drj', 'SilentGhost', 'docs@python', 'mbussonn', 'Anthony Sottile', 'mdk', 'xtreak']
pr_nums = ['13579', '15649', '15653']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue36853'
versions = ['Python 3.8']

[jaraco]

In working on some docs contributions, I've run into issues where docs builds are failing in CI differently than they're failing locally.

Locally, running make venv from Docs/ results in Sphinx 2, whereas on Azure Pipelines, the docs are built with Sphinx 1.8.2 (

cpython/.azure-pipelines/docs-steps.yml

Line 15 in 1d4b160

- script: python -m pip install sphinx==1.8.2 blurb python-docs-theme

).

When running with Sphinx 2, this error emerges when running make suspicious when a change is around that triggers suspicious code (such as this commit 9bde7fa):

Exception occurred:
File "/Users/jaraco/code/public/cpython/Doc/tools/extensions/suspicious.py", line 154, in report_issue
self.warn('[%s:%d] "%s" found in "%-.120s"' %
AttributeError: 'CheckSuspiciousMarkupBuilder' object has no attribute 'warn'

First, we probably want to make these processes consistent (define the dependencies in exactly one place).

Second, we should probably determine why the doc builds are failing on Sphinx 2 and update the suspicious code to support Sphinx 2.

[JulienPalard]

Hi Jaraco,

We're using sphinx 1.8 in production (docs.python.org) but we're ready for sphinx 2 so we should upgrade, and when we'll upgrade, we'll upgrade it everywhere.

You're right, make venv does not specify a version, at the sphinx 2 bump we'll have to pin it here too and try to get a single point of truth about which version we're using.

I've just locally ran sphinx 2.0.0 on 071cbd4 (the current tip of your PR) and I'm not getting any error, which one are you getting?

[SilentGhost]

Second, we should probably determine why the doc builds are failing on Sphinx 2 and update the suspicious code to support Sphinx 2.

They're failing because in sphinx 2 old and deprecated (since 1.6) custom logging API was removed. So all Builder.warn calls will have to be changed to use logging module (this applies to other similar methods, luckily they had the same names as logging methods).

[tirkarthi]

See also #13579 that replaces self.warn with self.logger.warn in Doc/tools/extensions/suspicious.py

[jaraco]

I've just locally ran sphinx 2.0.0 on 071cbd4 (the current tip of your PR) and I'm not getting any error, which one are you getting?

The issue occurs on 2347d3a with make suspicious with Sphinx 2.0.0 and 2.0.1.

Doc 2347d3ae36 $ make suspicious                                                                                                                                               
mkdir -p build
Building NEWS from Misc/NEWS.d with blurb
PATH=./venv/bin:$PATH sphinx-build -b suspicious -d build/doctrees -D latex_elements.papersize=  -W . build/suspicious 
Running Sphinx v2.0.1
loading ignore rules... done, 352 rules loaded
building [mo]: targets for 0 po files that are out of date
building [suspicious]: targets for 476 source files that are out of date
updating environment: 476 added, 0 changed, 0 removed
/Users/jaraco/code/public/cpython/Doc/tools/extensions/pyspecific.py:274: RemovedInSphinx30Warning: env.note_versionchange() is deprecated. Please use ChangeSetDomain.note_changeset() instead.
  env.note_versionchange('deprecated', version[0], node, self.lineno)
reading sources... [100%] whatsnew/index                                                                                                                                       
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [ 53%] library/importlib.metadata                                                                                                                            
Exception occurred:
  File "/Users/jaraco/code/public/cpython/Doc/tools/extensions/suspicious.py", line 154, in report_issue
    self.warn('[%s:%d] "%s" found in "%-.120s"' %
AttributeError: 'CheckSuspiciousMarkupBuilder' object has no attribute 'warn'
The full traceback has been saved in /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/sphinx-err-g3i65y10.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at </~https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make[1]: *** [build] Error 2
make: *** [suspicious] Error 1
Suspicious check complete; look for any errors in the above output or in build/suspicious/suspicious.csv.  If all issues are false positives, append that file to tools/susp-ignored.csv.

[drj11]

I believe the issue is only triggered if you actually have some suspicious markup in your documentation (which is why your plain build on Sphinx 2 appears to work).

Remove some lines from Doc/tools/susp-ignored.csv to trigger it.

[jaraco]

New changeset e1786b5 by Jason R. Coombs (Anthony Sottile) in branch 'master':
bpo-36853: Fix suspicious.py to actually print the unused rules (bpo-13579)
e1786b5

[jaraco]

New changeset b365cfa by Jason R. Coombs (Miss Islington (bot)) in branch '3.8':
bpo-36853: Fix suspicious.py to actually print the unused rules (GH-13579) (GH-15649)
b365cfa

[jaraco]

New changeset ebe709d by Jason R. Coombs (Anthony Sottile) in branch '3.7':
bpo-36853: Fix suspicious.py to actually print the unused rules (bpo-13579) (bpo-15653)
ebe709d

[hugovk]

Closing this old issue, make suspicious was deprecated last year and will be removed soon in favour of make check.

See #86404.