-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove the Quick testing framework #1240
Conversation
0cf4da3
to
59e6ac5
Compare
cababa8
to
25846ff
Compare
Spec/Stats.swift
Outdated
} | ||
|
||
enum TestCase_ReusableTestsTestDirection { | ||
case should_return_a_MessageTraffic_object |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(SwiftFormat will indent these cases, don't worry.)
Spec/Stats.swift
Outdated
} | ||
|
||
func test__002__Stats__all__should_return_value_for_message_counts() { | ||
test__Stats__all__reusableTestsTestAttribute(testCase: .should_return_value_for_message_counts) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(SwiftFormat will indent these statements, don't worry.)
expect(data.count) == 32 //32 bytes digest | ||
expect(data.count) == 32 | ||
afterEach() | ||
//32 bytes digest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Observation, doesn't block review)
The migrator has moved this comment to after the afterEach
call for some reason. I'll see if I can figure out why, but if it's a huge amount of work then I'll just fix this manually.
Spec/Auth.swift
Outdated
} | ||
|
||
context("with TTL") { | ||
|
||
// TE6 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Observation, doesn't block review)
The migrator has put this comment in the wrong place for some reason. It should be above the declaration of reusableTestsTestTokenRequestFromJson
. I'll see if I can figure out why, but if it's a huge amount of work then I'll just fix this manually.
This is the result of running ably/quick-to-xctest-migrator@fe10916 with --rewrite-locals-to-globals --no-rewrite-test-code.
This is the result of running ably/quick-to-xctest-migrator@fe10916 with no flags.
25846ff
to
e54234e
Compare
@lukasz-szyszkowski in the end I decided to implement your suggestion of using @maratal I made the |
What does this pull request do?
Removes the Quick testing framework.
Why are we doing this?
See #1201 for details. The main two reasons are:
How do we do it?
I wrote an automated migrator tool to do the bulk of the work here. (The code for the migrator is still rather messy. We're thinking about tidying it up to make it publicly available, but it's not there yet.)
How is this pull request structured?
Each of the steps below is a commit which I recommend reviewing individually (else you'll struggle to make any sense of this PR, I think):
spec
/describe
/context
level local variable and function declarations intoprivate
variables at the top of the file.QuickSpec
subclass into a subclass ofXCTestCase
. We replace thespec
method declaration with a set of reusable functions andfunc test…
instance methods.configuration.beforeSuite
hook.Does this change the behaviour of the tests?
For the most part, no. One of my main aims was for the migrated tests to behave exactly the same as the originals running under Quick. This turned out to be important, because we have tests that are dependent upon things like order of variable initialization and order of test execution (see my diff-level comments on this PR, and #1241).
The only difference is in the behaviour of local variables in the
reusableTests
functions. See this comment.How did I verify that the behaviour is unchanged?
The migrator has a
--add-logging
option, which adds logging statements to all of thebefore/afterEach
andit
calls. I used theably-cocoa-tests-linter-log-comparison
tool in the migrator repo to compare the logs emitted by the pre and post migration tests, and confirmed that everything runs in the same order.(The tool doesn't verify the order of variable initialization; that I did in a more ad-hoc way, adding a few logging statements selectively by hand.)
Notes
I've left comments on the PR for things that I wanted to point out, or which I'd like opinions on. Please ignore the "Outdated" tag for now; nothing is outdated yet 🙂