Testing Qt.py with VFX Platform CY2018 #234

fredrikaverpil · 2017-08-18T10:40:19Z

This is a work in progress and is not ready to merge.

Blocking:

To do:

Update DOCKER.md with CY2018 specification table
Update README.md
Update bash scripts to build, run and test using Docker
Address tests errors

fredrikaverpil · 2017-08-18T10:47:51Z

Tests errors

Python 2.7 error

======================================================================
FAIL: .wrapInstance doesn't need the `base` argument
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/python2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/usr/local/python2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/workdir/Qt.py/tests.py", line 501, in test_implicit_wrapInstance
    assert widget != button
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
Order: 'PySide2'
Trying PySide2
'QtCore.QMetaType' was missing.
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------

Python 3.x error

======================================================================
FAIL: Doctest: test_caveats.test_11_qtcompat_wrapinstance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/python3.6/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/local/python3.6/lib/python3.6/unittest/case.py", line 605, in run
    testMethod()
  File "/usr/local/python3.6/lib/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test_11_qtcompat_wrapinstance
  File "/workdir/Qt.py/test_caveats.py", line 175, in test_11_qtcompat_wrapinstance
----------------------------------------------------------------------
File "/workdir/Qt.py/test_caveats.py", line 191, in test_11_qtcompat_wrapinstance
Failed example:
    widget == button
Expected:
    False
Got:
    True

fredrikaverpil · 2017-08-21T11:13:35Z

Here we go: https://codereview.qt-project.org/gitweb?p=pyside/pyside-setup.git;a=commit;h=8aca0dec116c6b321373f5636b34036530ac6e4b

__version__         = "2.0.0~alpha0"
__version_info__    = (2, 0, 0, "alpha", 0)

__build_date__ = '2017-08-25T22:36:31+00:00'
__build_commit_date__ = '2017-08-24T14:05:49+00:00'

__build_commit_hash__ = '117e0ff91275b4bc06dd5383f19e7028c5ef6ff8'
__build_commit_hash_described__ = '2.0.0.dev0-4859-g117e0ff9'

__build_commit_date__ seems to be the most safe one to use.

…rm2018

fredrikaverpil · 2017-08-26T07:58:28Z

@mottosso so, now I need to do some checking in our tests whether a new PySide2 is used or an older one.

I'm thinking we could add a new function to QtCompat which parses the PySide2.__build_commit_date__ into a datetime format:

import datetime

# Just example code...
def pyside2_commit_date():
    try:
        commit_date = PySide2.__build_commit_date__
        datetime_object = datetime.strptime(
            commit_date[: commit_date.rfind('+')], '%Y-%m-%dT%H:%M:%S'
        )
    except AttributeError:
        # PySide2 version which doesn't have the attribute
        datetime_object = None
    return datetime_object

In our tests, we could then do something like:

import datetime

pyside2_commit_date = QtCompat.pyside2_commit_date()
if pyside2_commit_date is None:
    # old version is being used
elif pyside2_commit_date <= SOME_DATE:
    # pyside2 is older or equal to SOME_DATE
...
...

Note: I just wrote this code without having run it. So, it's a bit like pseudo at this point.

This way we can utilize date checks against PySide2 builds in tests, caveats – and make the tests code more grokable, I think. What do you think?

fredrikaverpil · 2017-08-26T15:39:58Z

I ended up not adding anything to QtCompat. Instead I added a _pyside2_commit_date() function to tests.py (EDIT: and build_membership.py). So, inside tests.py, we can now query the git commit date of PySide2. Right now, I'm just querying whether this attribute exists or not but in the future we can compare dates more explicitly.

I don't think I can put if conditions in the caveats code snippets and then also put a return value, can I?
Like this I mean...

>>> if hasattr(Qt, '__build_commit_date__'):
...     widget == button
True
>>> else:
...     widget == button
False

Anyway, I ended up adding untested and a note instead to the erroring caveat.

Now, we're getting a lot of membership errors with Python 2.7. But all other platforms/combos are green/OK. However, if they were to run the membership tests, they would also fail. Not sure how we should proceed here... should we add more members to the excluded dictionary in build_membership.py or should I create a new dictionary which is intended for this version of PySide2?

fredrikaverpil · 2017-09-24T15:25:24Z

Common member changes; topics for discussion

This PR is getting ready to be merged. But...

We really haven't talked about how to deal with the fact that PySide2 seems to be picking up speed and is adding new members - and how Qt.py should reflect this change.

Background

Right now, we have a script in place (run_membership.sh) which will generate the common members of the bindings used in our CY2017 Docker test container. The binding versions used in the container doesn't get updated. Instead we add a new container each year (around time of SIGGRAPH) where binding versions may or may not get updated (which is what I'm doing here in this PR). So in this PR I'm adding a more recent version of PySide2 and the membership script detects new members, previously not common across bindings.

So how do we deal with a change of common members?
The way I see it, we have some options depending how we want to play this.

Clarifying our decisions on this in the project goals

Either we just go with what's the latest and greatest and just include these new common members in Qt.py. Or we do something different. The drawbacks of going with the latest common bindings generated using the latest container is that those members are not available in older versions of PySide2 (which may exist in old versions of e.g. Maya, Nuke etc). This, to me seemingly new, behaviour is not completely clear in e.g. our project goals.

I think it would be multiple orders of difficult to maintain a Qt.py which is somehow aware of where it is running and what limitations in terms of common members it may have. I think we should just go with the latest and greatest. This means slightly tweaking and clarifying this in the project goals, I think.

Dealing with this in our membership test

Right now, we have a test for members. In this test, I'm excluding members based on which container is running the test. This seems to me to be the most pragmatic way to make this test run. When new members are added to Qt.py, old containers would fail and we'd have to add the new members to a "skip list" in the membership test. Right now, this was the most maintainable approach I could come up with.

Documenting the member changes

I wouldn't call the "skip list" in the membership test a documentation of common members change ..but it kind of is, for us. I don't think this is worth dragging out from the darkness (of the tests) and into the light (of the main README.md) and present such changes there. Especially not since everyone is probably using different versions of PySide2 (without being able to pinpoint the exact version due to the lack of a version string).

The need to update the Docker containers more frequently

It's likely that a user is running a later build of PySide2 than we have in our Docker test container, and that they believe a certain member needs to be added to Qt.py. We could potentially build a new Docker image and append this to the current set of test containers. But I'd say we postpone this issue onto the future for when (and if) it happens. I have great hopes that we won't have to update the test containers more often than once a year.

@mottosso what do you think about all of this?

mottosso · 2017-09-24T18:24:33Z

The need to update the Docker containers more frequently

They won't need any updates, so long as they are an accurate reflection of the versions set by VFX Reference Platform. Users using any other versions are on their own.

At most, we'll need a new image each year. I think that should answer the remaining questions too?

fredrikaverpil · 2017-09-24T21:39:37Z

The problem lies with that PySide2 does not have a maintained version string (PySide2 built from any git commit from the "5.6" branch has version "2.0.0~alpha0") and the VFX Platform just specifies "PySide 2.0.x" which basically translates into any version (or commit) of PySide2.

Users using any other versions are on their own.

That's everyone, including us. Everyone probably just use the latest source available at the time when they need to build PySide2. We chose to use a build around the time of SIGGRAPH.

The VFX Platform could specify a git commit or a git commit date/hash, but they aren't doing that.

This is why I posted my previous novel-like "topics for discussion"...

fredrikaverpil · 2017-09-25T06:59:10Z

To avoid getting too deep into complex discussions over theoretical scenarios, I've instead updated the README.md and DOCKER.md to attempt to explain the issue at hand.

So, in case you understand what I'm trying to explain I think we are good :) but if you find the updated README (check the diff) hard to grok, then we probably need to clarify this further.

@mottosso To sum it up, it would be great with a review of this PR and add the following to the next release notes:

New common members added:
  - QtGui.QDesktopServices
  - QtMultimedia
  - QtOpenGL

I think we should then close #219 and consider releasing a non-beta version of Qt.py at PyPi and in Anaconda cloud.

albertosottile · 2017-10-02T12:17:41Z

Since you are adding modules, it would also be useful to include QtCore.QStandardPaths in the next release. Thanks.

fredrikaverpil · 2017-10-02T14:14:07Z

@alby128 I don't see QtCore.QStandardPaths in either of the bindings/versions (Qt <=5.6). Which binding are you using where this is available and which version?

Qt.py will only contain built in support for members available throughout all bindings. You can modify this behavior and add members yourself using QtSiteConfig.py. Please refer to the main README for more information.

albertosottile · 2017-10-02T14:34:54Z

QStandardPaths has been introduced in Qt 5 as (partial) replacement for (some features of) QDesktopServices. See the first paragraph here: http://doc.qt.io/qt-5.6/whatsnew50.html . So it is not available on PySide1.x and PyQt4. However, it is available on PyQt5 (see here) and on PySide2 (I checked in a version that I just built from sources and in the conda binaries).

I do not know what is your policy for modules that are only available on Qt >= 5.6. Thanks for the tip on how to add a member.

fredrikaverpil · 2017-10-02T15:37:15Z

@alby128 thanks for clarifying. We want Qt.py to support members which exist throughout all the bindings so that software built with Qt.py has greater chances running regardless of binding. We're know this limits the Qt library compatibility but this is also one of the reasons we built Qt.py to begin with; to support both Qt4 and Qt5 bindings (but only members which exists in both).

Of course, your request is not uncommon and that's why we also enabled the possibility to add your own members in case you need to break out of "total compatibility". I hope this works for you (QtSiteConfig.py) as we don't want to add these specific members to Qt.py as they do not exist throughout all the bindings.

mottosso · 2017-12-02T09:56:21Z

Hey @fredrikaverpil, should we merge this?

fredrikaverpil · 2017-12-05T06:40:32Z

@mottosso yes, we should be fine merging this. Two changes were made to the original CY2018 which isn't part of this docker build:

26th November 2017 - CY2018 has had a late change to the compiler version which is now gcc 6.3.1. This successfully resolves issues that were discovered with gcc 5.3.1.

28th August 2017 - Added a note for gcc 5 and updated the Qt note to include a link to the qtdeclarative modifications.

I'll try to cover those as soon as I can, but in the meanwhile we're fine merging this as-is.

fredrikaverpil · 2017-12-05T06:41:03Z

I'm re-running the latest checks and when done, feel free to merge!

fredrikaverpil · 2017-12-05T16:28:34Z

All tests OK - merge away!

PS. I noticed that Qt4 (required for PySide, PyQt4) won't compile with the late CY2018 gcc version change (gcc 6.3.1). This means we'll have to manage two gcc versions for a CY2018-Update 1 image. But for now, let's merge this.

fredrikaverpil · 2017-12-11T08:00:09Z

How about bumping the version to 1.1.0b8 and planning for a 1.1.0 release on pip and anaconda cloud?
Do you think we should finish some other stuff prior to releasing 1.1.0?

mottosso · 2017-12-11T08:13:43Z

Sounds like a good idea. 👍

Add dockerfile for CY2018 and include in Travis tests

0cf893b

fredrikaverpil self-assigned this Aug 18, 2017

fredrikaverpil mentioned this pull request Aug 19, 2017

News: A more complete PySide2 for CentOS 6 build exists in the Qt.py project fredrikaverpil/pyside2-wheels#80

Closed

Add note on entering the container, bypassing entrypoint

a5e9aa7

fredrikaverpil added 13 commits August 24, 2017 07:45

Merge remote-tracking branch 'upstream/master' into feature/vfxplatfo…

d66bb7c

…rm2018

Define PySide2 commit SHA

e095431

Add pyside2-uic bug workaround

07bc838

Define commit SHAs for Qt5 and qt-creator

50bc737

Use new fredrikaverpil/qt.py:2018 image

b4f551a

Clean up

164c6fd

Try different commit SHA

aa713a3

Try yet another Qt5 git commit hash

86bcb32

Use latest commits of Qt5, adsk-qtbase, adsk-qtx11extras

fa79eed

Use latest commits for Qt 5.6.1 and PySide2

b17325e

Remove workaround for PYSIDE-552

47f7fc8

Fix wrong variable

89d3cc1

Fix wrong repo url

db5291b

fredrikaverpil added 5 commits August 26, 2017 17:11

Add VFX Platform reference CY2018

858a729

Improve command to get Qt.py copied into workdir

cca8dd6

Implement _pyside2_commit_date()

f28da56

Skip caveat and add note

3145c43

Remove test code

8e2ce3c

fredrikaverpil added 4 commits August 26, 2017 21:53

Be more explicit

1c94539

Reset the exclude dictionary for PySide2 as of 2017-08-25

a6f5d58

Update date

5d98438

Fix calendar year

bcdc86e

Skip some members for CY2017

4a4310c

fredrikaverpil added 8 commits September 25, 2017 08:37

Remove old build_membership.py script

966771f

Rename bash script

7baa7d5

Revise explanation of software versions

ea1ffe6

Remove mentioning of membership tests and revise subset explanation

db7c5f4

Update docstring

59ffd7b

Fix typo

efc5f19

Update docstring

fc25f6c

Improve on explaining how the common members are generated

38e1105

fredrikaverpil requested a review from mottosso September 25, 2017 06:59

Add better clarification

be1cd24

fredrikaverpil mentioned this pull request Sep 25, 2017

QDesktopServices in Qt.py #219

Closed

fredrikaverpil added 2 commits September 25, 2017 09:15

Add note on 5.6 branch

05c9c1a

Add reference to QSiteConfig

11f260b

mottosso merged commit 1f3f9b7 into mottosso:master Dec 9, 2017

fredrikaverpil mentioned this pull request Dec 11, 2017

Bump version to 1.1.0.b8 #259

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Testing Qt.py with VFX Platform CY2018 #234

Testing Qt.py with VFX Platform CY2018 #234

fredrikaverpil commented Aug 18, 2017 •

edited

Loading

fredrikaverpil commented Aug 18, 2017 •

edited

Loading

fredrikaverpil commented Aug 21, 2017 •

edited

Loading

fredrikaverpil commented Aug 26, 2017 •

edited

Loading

fredrikaverpil commented Aug 26, 2017 •

edited

Loading

fredrikaverpil commented Sep 24, 2017 •

edited

Loading

mottosso commented Sep 24, 2017

fredrikaverpil commented Sep 24, 2017

fredrikaverpil commented Sep 25, 2017 •

edited

Loading

albertosottile commented Oct 2, 2017

fredrikaverpil commented Oct 2, 2017 •

edited

Loading

albertosottile commented Oct 2, 2017 •

edited

Loading

fredrikaverpil commented Oct 2, 2017 •

edited

Loading

mottosso commented Dec 2, 2017

fredrikaverpil commented Dec 5, 2017 •

edited

Loading

fredrikaverpil commented Dec 5, 2017 •

edited

Loading

fredrikaverpil commented Dec 5, 2017 •

edited

Loading

fredrikaverpil commented Dec 11, 2017 •

edited

Loading

mottosso commented Dec 11, 2017

Testing Qt.py with VFX Platform CY2018 #234

Testing Qt.py with VFX Platform CY2018 #234

Conversation

fredrikaverpil commented Aug 18, 2017 • edited Loading

fredrikaverpil commented Aug 18, 2017 • edited Loading

Tests errors

Python 2.7 error

Python 3.x error

fredrikaverpil commented Aug 21, 2017 • edited Loading

fredrikaverpil commented Aug 26, 2017 • edited Loading

fredrikaverpil commented Aug 26, 2017 • edited Loading

fredrikaverpil commented Sep 24, 2017 • edited Loading

Common member changes; topics for discussion

Background

Clarifying our decisions on this in the project goals

Dealing with this in our membership test

Documenting the member changes

The need to update the Docker containers more frequently

mottosso commented Sep 24, 2017

fredrikaverpil commented Sep 24, 2017

fredrikaverpil commented Sep 25, 2017 • edited Loading

albertosottile commented Oct 2, 2017

fredrikaverpil commented Oct 2, 2017 • edited Loading

albertosottile commented Oct 2, 2017 • edited Loading

fredrikaverpil commented Oct 2, 2017 • edited Loading

mottosso commented Dec 2, 2017

fredrikaverpil commented Dec 5, 2017 • edited Loading

fredrikaverpil commented Dec 5, 2017 • edited Loading

fredrikaverpil commented Dec 5, 2017 • edited Loading

fredrikaverpil commented Dec 11, 2017 • edited Loading

mottosso commented Dec 11, 2017

fredrikaverpil commented Aug 18, 2017 •

edited

Loading

fredrikaverpil commented Aug 18, 2017 •

edited

Loading

fredrikaverpil commented Aug 21, 2017 •

edited

Loading

fredrikaverpil commented Aug 26, 2017 •

edited

Loading

fredrikaverpil commented Aug 26, 2017 •

edited

Loading

fredrikaverpil commented Sep 24, 2017 •

edited

Loading

fredrikaverpil commented Sep 25, 2017 •

edited

Loading

fredrikaverpil commented Oct 2, 2017 •

edited

Loading

albertosottile commented Oct 2, 2017 •

edited

Loading

fredrikaverpil commented Oct 2, 2017 •

edited

Loading

fredrikaverpil commented Dec 5, 2017 •

edited

Loading

fredrikaverpil commented Dec 5, 2017 •

edited

Loading

fredrikaverpil commented Dec 5, 2017 •

edited

Loading

fredrikaverpil commented Dec 11, 2017 •

edited

Loading