Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear app data before all e2e tests #1632

Merged
merged 5 commits into from
Mar 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import android.widget.Button
import android.widget.EditText
import com.bugsnag.android.Configuration
import com.bugsnag.android.mazerunner.scenarios.Scenario
import java.io.File

class MainActivity : Activity() {

Expand All @@ -30,6 +31,15 @@ class MainActivity : Activity() {
sendBroadcast(closeDialog)
}

// Clear persistent data (used to stop scenarios bleeding into each other)
findViewById<Button>(R.id.clear_persistent_data).setOnClickListener {
clearFolder("last-run-info")
clearFolder("bugsnag-errors")
clearFolder("device-id")
clearFolder("user-info")
clearFolder("fake")
}

// load the scenario first, which initialises bugsnag without running any crashy code
findViewById<Button>(R.id.start_bugsnag).setOnClickListener {
scenario = loadScenarioFromUi()
Expand Down Expand Up @@ -73,6 +83,13 @@ class MainActivity : Activity() {
}
}

private fun clearFolder(name: String) {
val context = MazerunnerApp.applicationContext()
val folder = File(context.cacheDir, name)
log("Clearing folder: ${folder.path}")
folder.deleteRecursively()
}

private fun loadScenarioFromUi(): Scenario {
val scenarioPicker = findViewById<EditText>(R.id.scenario_name)
val eventType = scenarioPicker.text.toString()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,24 @@
package com.bugsnag.android.mazerunner

import android.app.Application
import android.content.Context
import android.os.Build
import android.os.StrictMode

class MazerunnerApp : Application() {

init {
instance = this
}

companion object {
private var instance: MazerunnerApp? = null

fun applicationContext(): Context {
return instance!!.applicationContext
}
}

override fun onCreate() {
super.onCreate()
triggerStartupAnrIfRequired()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@
android:hint="Scenario metadata"
android:inputType="text" />

<Button
android:id="@+id/clear_persistent_data"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_weight="0"
android:text="Clear app data" />

<Button
android:id="@+id/start_bugsnag"
android:layout_width="match_parent"
Expand Down
21 changes: 12 additions & 9 deletions features/full_tests/auto_notify.feature
Original file line number Diff line number Diff line change
@@ -1,28 +1,31 @@
Feature: Switching automatic error detection on/off for Unity

Scenario: Handled JVM exceptions are captured with autoNotify=false
Background:
Given I clear all persistent data

Scenario: Handled JVM exceptions are captured with autoNotify=false
When I run "HandledJvmAutoNotifyFalseScenario"
Then I wait to receive an error
And the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
And the exception "message" equals "HandledJvmAutoNotifyFalseScenario"

Scenario: JVM exception not captured with autoNotify=false
Scenario: JVM exception not captured with autoNotify=false
When I run "UnhandledJvmAutoNotifyFalseScenario" and relaunch the app
And I configure Bugsnag for "UnhandledJvmAutoNotifyFalseScenario"
Then Bugsnag confirms it has no errors to send

Scenario: NDK signal not captured with autoNotify=false
Scenario: NDK signal not captured with autoNotify=false
When I run "UnhandledNdkAutoNotifyFalseScenario" and relaunch the app
And I configure Bugsnag for "UnhandledNdkAutoNotifyFalseScenario"
Then Bugsnag confirms it has no errors to send

@skip_android_8_1
Scenario: ANR not captured with autoDetectAnrs=false
@skip_android_8_1
Scenario: ANR not captured with autoDetectAnrs=false
When I run "AutoDetectAnrsFalseScenario" and relaunch the app
And I configure Bugsnag for "AutoDetectAnrsFalseScenario"
Then Bugsnag confirms it has no errors to send

Scenario: JVM exception captured with autoNotify reenabled
Scenario: JVM exception captured with autoNotify reenabled
When I run "UnhandledJvmAutoNotifyTrueScenario" and relaunch the app
And I configure Bugsnag for "UnhandledJvmAutoNotifyTrueScenario"
Then I wait to receive an error
Expand All @@ -34,7 +37,7 @@ Scenario: JVM exception captured with autoNotify reenabled
And the event "unhandled" is true
And the event "severity" equals "error"

Scenario: NDK signal captured with autoNotify reenabled
Scenario: NDK signal captured with autoNotify reenabled
When I run "UnhandledNdkAutoNotifyTrueScenario" and relaunch the app
And I configure Bugsnag for "UnhandledNdkAutoNotifyTrueScenario"
Then I wait to receive an error
Expand All @@ -48,8 +51,8 @@ Scenario: NDK signal captured with autoNotify reenabled
And the event "severityReason.unhandledOverridden" is false

# PLAT-6620
@skip_android_8_1
Scenario: ANR captured with autoDetectAnrs reenabled
@skip_android_8_1
Scenario: ANR captured with autoDetectAnrs reenabled
When I run "AutoDetectAnrsTrueScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand Down
11 changes: 7 additions & 4 deletions features/full_tests/breadcrumb.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: Reporting Breadcrumbs

Scenario: Manually added breadcrumbs are sent in report
Background:
Given I clear all persistent data

Scenario: Manually added breadcrumbs are sent in report
When I run "BreadcrumbScenario"
And I wait to receive an error
Then the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
Expand All @@ -17,19 +20,19 @@ Scenario: Manually added breadcrumbs are sent in report
And the event "breadcrumbs.0.name" equals "Hello Breadcrumb!"
And the event "breadcrumbs.0.type" equals "manual"

Scenario: Manually added breadcrumbs are sent in report when auto breadcrumbs are disabled
Scenario: Manually added breadcrumbs are sent in report when auto breadcrumbs are disabled
When I run "BreadcrumbDisabledScenario"
And I wait to receive an error
Then the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
And the event has 1 breadcrumbs

Scenario: An automatic breadcrumb is sent in report when the appropriate type is enabled
Scenario: An automatic breadcrumb is sent in report when the appropriate type is enabled
When I run "BreadcrumbAutoScenario"
And I wait to receive an error
Then the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
And the event has a "state" breadcrumb with the message "Bugsnag loaded"

Scenario: Error Breadcrumbs appear in subsequent events
Scenario: Error Breadcrumbs appear in subsequent events
When I run "ErrorBreadcrumbsScenario" and relaunch the app
And I configure Bugsnag for "ErrorBreadcrumbsScenario"
Then I wait to receive 2 errors
Expand Down
5 changes: 4 additions & 1 deletion features/full_tests/bugsnag_init.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: Verify the Bugsnag Initialization methods

Scenario: Test Bugsnag initializes correctly
Background:
Given I clear all persistent data

Scenario: Test Bugsnag initializes correctly
When I run "BugsnagInitScenario"
And I wait to receive an error
Then the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
Expand Down
5 changes: 4 additions & 1 deletion features/full_tests/crash_handler.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: Reporting with other exception handlers installed

Scenario: Other uncaught exception handler installed
Background:
Given I clear all persistent data

Scenario: Other uncaught exception handler installed
When I run "CrashHandlerScenario" and relaunch the app
And I configure Bugsnag for "CrashHandlerScenario"
And I wait to receive an error
Expand Down
7 changes: 5 additions & 2 deletions features/full_tests/custom_http_client.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: Using custom API clients for reporting errors

Scenario: Set a custom HTTP client and flush a stored error + session
Background:
Given I clear all persistent data

Scenario: Set a custom HTTP client and flush a stored error + session
When I configure the app to run in the "offline" state
And I run "CustomHttpClientFlushScenario" and relaunch the app
And I configure Bugsnag for "CustomHttpClientFlushScenario"
Expand All @@ -15,7 +18,7 @@ Scenario: Set a custom HTTP client and flush a stored error + session
And the session "Custom-Client" header equals "Hello World"
And the session is valid for the session reporting API version "1.0" for the "Android Bugsnag Notifier" notifier

Scenario: Set a custom HTTP client and send an error + session
Scenario: Set a custom HTTP client and send an error + session
When I run "CustomHttpClientScenario"

# error received
Expand Down
7 changes: 5 additions & 2 deletions features/full_tests/detect_anr_cxx.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: ANRs triggered in CXX code are captured

Scenario: ANR triggered in CXX code is captured
Background:
Given I clear all persistent data

Scenario: ANR triggered in CXX code is captured
When I run "CXXAnrScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand All @@ -14,7 +17,7 @@ Scenario: ANR triggered in CXX code is captured
And the error payload field "events.0.threads.0.type" equals "android"
And the error payload field "events.0.threads.0.stacktrace.0.type" is null

Scenario: ANR triggered in CXX code is captured even when NDK detection is disabled
Scenario: ANR triggered in CXX code is captured even when NDK detection is disabled
When I run "CXXAnrNdkDisabledScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand Down
13 changes: 8 additions & 5 deletions features/full_tests/detect_anr_jvm.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: ANRs triggered in JVM code are captured

Scenario: ANR triggered in JVM loop code is captured
Background:
Given I clear all persistent data

Scenario: ANR triggered in JVM loop code is captured
When I run "JvmAnrLoopScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand All @@ -16,8 +19,8 @@ Scenario: ANR triggered in JVM loop code is captured

# Other scenarios use a deadlock to generate an ANR, which works on Samsung devices. This scenario remains skipped
# on Samsung as it is explicitly design to test ANRs caused by a sleeping thread.
@skip_samsung
Scenario: ANR triggered in JVM sleep code is captured
@skip_samsung
Scenario: ANR triggered in JVM sleep code is captured
When I run "JvmAnrSleepScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand All @@ -30,7 +33,7 @@ Scenario: ANR triggered in JVM sleep code is captured
And the error "Bugsnag-Stacktrace-Types" header equals "android,c"
And the error payload field "events.0.exceptions.0.type" equals "android"

Scenario: ANR triggered in JVM code is not captured when detectAnrs = false
Scenario: ANR triggered in JVM code is not captured when detectAnrs = false
When I run "JvmAnrDisabledScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand All @@ -40,7 +43,7 @@ Scenario: ANR triggered in JVM code is not captured when detectAnrs = false
And the exception "errorClass" equals "java.lang.RuntimeException"
And the exception "message" equals "JvmAnrDisabledScenario"

Scenario: ANR triggered in JVM code is not captured when outside of release stage
Scenario: ANR triggered in JVM code is not captured when outside of release stage
When I run "JvmAnrOutsideReleaseStagesScenario"
And I wait for 2 seconds
And I tap the screen 3 times
Expand Down
5 changes: 4 additions & 1 deletion features/full_tests/detect_ndk_crashes.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: Verifies autoDetectNdkCrashes controls when NDK crashes are reported

Scenario: No crash reported when autoDetectNdkCrashes disabled
Background:
Given I clear all persistent data

Scenario: No crash reported when autoDetectNdkCrashes disabled
When I run "AutoDetectNdkDisabledScenario" and relaunch the app
And I configure Bugsnag for "AutoDetectNdkDisabledScenario"
And I wait for 2 seconds
Expand Down
5 changes: 4 additions & 1 deletion features/full_tests/empty_stacktrace.feature
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Feature: Empty Stacktrace reported

Scenario: Exceptions with empty stacktraces are sent
Background:
Given I clear all persistent data

Scenario: Exceptions with empty stacktraces are sent
When I run "EmptyStacktraceScenario"
Then I wait to receive an error
And the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
Expand Down
7 changes: 5 additions & 2 deletions features/full_tests/event_callback_alters_api_key.feature
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
Feature: When the api key is altered in an Event the JSON payload reflects this

Scenario: Handled exception with altered API key
Background:
Given I clear all persistent data

Scenario: Handled exception with altered API key
When I run "HandledExceptionApiKeyChangeScenario"
Then I wait to receive an error
And the error payload field "events" is an array with 1 elements
And the exception "message" equals "HandledExceptionApiKeyChangeScenario"
And the error payload field "apiKey" equals "0000111122223333aaaabbbbcccc9999"
And the error "Bugsnag-Api-Key" header equals "0000111122223333aaaabbbbcccc9999"

Scenario: Unhandled exception with altered API key
Scenario: Unhandled exception with altered API key
When I run "UnhandledExceptionApiKeyChangeScenario" and relaunch the app
And I configure Bugsnag for "UnhandledExceptionApiKeyChangeScenario"
And I wait to receive an error
Expand Down
Loading