Update testing

[elliottower]

Description

Splitting #933 into separate PRs for simplicity.

This PR rewrites the seed test to actually check that environment rollouts are the same, matching gymnasium. It also updates the API test to handle dict observations, and fixes the variable env test which fails under the new seed test code.

The previous seed test used hashing to check that observations and rewards were consistent, but was nearly impossible to debug (all you see are hash values), not comprehensive (didn't check for many things like info matching, agents being the same, etc) and generally very confusing/messy code. The only reason waterworld passed previously was that the seeds used were 0,1,2, whereas seed 42 which I use (same as gymnasium/shimmy) exposed underlying issues in the environment. I'm keeping the seed at 42 and disabling the waterworld seed test, so it is documented and the bug can be tested and fixed in the future.

It also fixes a bug in one environment where obs and obs space don't match.

Fixes # (issue), Depends on # (pull request)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.
To upload images to a PR -- simply drag and drop or copy paste.

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have run pytest -v and no errors are present.
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[jjshoots]

@elliottower

So I've been poking around at this, it appears that

1. When this is called,
2. this gets triggered, which creates a new agent (here called type for whatever confusing reason)
3. which adds new observation and action spaces for the new agent here.
4. The problem is that these spaces are not seeded, which causes the tests to fail.

The solution here is to simply:

1. Store the seed in reset using something like self.rng_seed = seed.
2. Under add_type, do obs_space.seed(self.rng_seed) and act_space.seed(self.rng_seed) after the obs and action spaces are declared.

[elliottower]

Note: looking at rlcard base, it seems that even if an env uses np float64 it gets recast to float32, so I decided to keep my original change making the observation space float32 (which also caused the tests not to fail)

[jjshoots]

Sorry, but some last final knitpicks, but otherwise golden.