venv_backend new options and choices (#316)

* New global option `nox.options.venv_backend` to set the default backend. Fixes #315

* Added doc about the new option

* Blackened

* fixed tests

* Fixed docs

* Fixed coverage by adding tests for the venv_backend_completer

* fixed test

* Added tests for short and long versions of the new option.

* Replaced the venv_backend completer with a simple `choices` from argparse :)

* Renamed venv_backend to default_venv_backend, and created new force_venv_backend

* New "none" choice for venv_backends, equivalent to python=False

* Updated doc concerning default_venv_backend and force_venv_backend, as well as the new 'none' backend

* Fixed all manifest tests

* Fixed test_tasks for venv_backend

* Fixed coverage

* Blackened code

* The warning message was appearing for all sessions, even those deselected. It is now only logged when session is run.

* Added `--no-venv` option. Fixes #301

* Blackened

* Fixed tests

* Improved coverage

* Blackened

* Fixed an issue with parametrization: warning would not be issued. Added corresponding tests. This should make coverage happy, too.

* Blackened

* Now `install` and `conda_install` work when there is no venv backend (or python=False). Previously a `ValueError` was raised. Fixes #318

* Fixed test

* Minor edit to trigger CI build again as it seems stuck.

* Minor doc fix to trigger the CI again (appveyor false fail)

Co-authored-by: Sylvain MARIE <sylvain.marie@se.com>

docs/config.rst

@@ -132,7 +132,7 @@ Will produce these sessions:
 
 Note that this expansion happens *before* parameterization occurs, so you can still parametrize sessions with multiple interpreters.
 
-If you want to disable virtualenv creation altogether, you can set ``python`` to ``False``:
+If you want to disable virtualenv creation altogether, you can set ``python`` to ``False``, or set ``venv_backend`` to ``"none"``, both are equivalent. Note that this can be done temporarily through the :ref:`--no-venv <opt-force-venv-backend>` commandline flag, too.
 
 .. code-block:: python
 
@@ -375,6 +375,8 @@ The following options can be specified in the Noxfile:
 * ``nox.options.sessions`` is equivalent to specifying :ref:`-s or --sessions <opt-sessions-pythons-and-keywords>`.
 * ``nox.options.pythons`` is equivalent to specifying :ref:`-p or --pythons <opt-sessions-pythons-and-keywords>`.
 * ``nox.options.keywords`` is equivalent to specifying :ref:`-k or --keywords <opt-sessions-pythons-and-keywords>`.
+* ``nox.options.default_venv_backend`` is equivalent to specifying :ref:`-db or --default-venv-backend <opt-default-venv-backend>`.
+* ``nox.options.force_venv_backend`` is equivalent to specifying :ref:`-fb or --force-venv-backend <opt-force-venv-backend>`.
 * ``nox.options.reuse_existing_virtualenvs`` is equivalent to specifying :ref:`--reuse-existing-virtualenvs <opt-reuse-existing-virtualenvs>`. You can force this off by specifying ``--no-reuse-existing-virtualenvs`` during invocation.
 * ``nox.options.stop_on_first_error`` is equivalent to specifying :ref:`--stop-on-first-error <opt-stop-on-first-error>`. You can force this off by specifying ``--no-stop-on-first-error`` during invocation.
 * ``nox.options.error_on_missing_interpreters`` is equivalent to specifying :ref:`--error-on-missing-interpreters <opt-error-on-missing-interpreters>`. You can force this off by specifying ``--no-error-on-missing-interpreters`` during invocation.

docs/usage.rst

@@ -105,12 +105,51 @@ Then running ``nox --session tests`` will actually run all parametrized versions
     nox --session "tests(django='2.0')"
 
 
+.. _opt-default-venv-backend:
+
+Changing the sessions default backend
+-------------------------------------
+
+By default nox uses ``virtualenv`` as the virtual environment backend for the sessions, but it also supports ``conda`` and ``venv`` as well as no backend (passthrough to whatever python environment nox is running on). You can change the default behaviour by using ``-db <backend>`` or ``--default-venv-backend <backend>``. Supported names are ``('none', 'virtualenv', 'conda', 'venv')``.
+
+.. code-block:: console
+
+    nox -db conda
+    nox --default-venv-backend conda
+
+
+You can also set this option in the Noxfile with ``nox.options.default_venv_backend``. In case both are provided, the commandline argument takes precedence.
+
+Note that using this option does not change the backend for sessions where ``venv_backend`` is explicitly set.
+
+
+.. _opt-force-venv-backend:
+
+Forcing the sessions backend
+----------------------------
+
+You might work in a different environment than a project's default continuous integration setttings, and might wish to get a quick way to execute the same tasks but on a different venv backend. For this purpose, you can temporarily force the backend used by **all** sessions in the current nox execution by using ``-fb <backend>`` or ``--force-venv-backend <backend>``. No exceptions are made, the backend will be forced for all sessions run whatever the other options values and nox file configuration. Supported names are ``('none', 'virtualenv', 'conda', 'venv')``.
+
+.. code-block:: console
+
+    nox -fb conda
+    nox --force-venv-backend conda
+
+
+You can also set this option in the Noxfile with ``nox.options.force_venv_backend``. In case both are provided, the commandline argument takes precedence.
+
+Finally note that the ``--no-venv`` flag is a shortcut for ``--force-venv-backend none`` and allows to temporarily run all selected sessions on the current python interpreter (the one running nox).
+
+.. code-block:: console
+
+    nox --no-venv
+
 .. _opt-reuse-existing-virtualenvs:
 
 Re-using virtualenvs
 --------------------
 
-By default nox deletes and recreates virtualenvs every time it is run. This is usually fine for most projects and continuous integration environments as `pip's caching <https://pip.pypa.io/en/stable/reference/pip_install/#caching>`_ makes re-install rather quick. However, there are some situations where it is advantageous to re-use the virtualenvs between runs. Use ``-r`` or ``--reuse-existing-virtualenvs``:
+By default, Nox deletes and recreates virtualenvs every time it is run. This is usually fine for most projects and continuous integration environments as `pip's caching <https://pip.pypa.io/en/stable/reference/pip_install/#caching>`_ makes re-install rather quick. However, there are some situations where it is advantageous to re-use the virtualenvs between runs. Use ``-r`` or ``--reuse-existing-virtualenvs``:
 
 .. code-block:: console
 

nox/_decorators.py

@@ -1,7 +1,7 @@
 import copy
 import functools
 import types
-from typing import Any, Callable, Iterable, List, Optional, cast
+from typing import Any, Callable, Iterable, List, Dict, Optional, cast
 
 from . import _typing
 
@@ -40,12 +40,14 @@ def __init__(
         name: Optional[str] = None,
         venv_backend: Any = None,
         venv_params: Any = None,
+        should_warn: Dict[str, Any] = None,
     ):
         self.func = func
         self.python = python
         self.reuse_venv = reuse_venv
         self.venv_backend = venv_backend
         self.venv_params = venv_params
+        self.should_warn = should_warn or dict()
 
     def __call__(self, *args: Any, **kwargs: Any) -> Any:
         return self.func(*args, **kwargs)
@@ -58,6 +60,7 @@ def copy(self, name: str = None) -> "Func":
             name,
             self.venv_backend,
             self.venv_params,
+            self.should_warn,
         )
 
 
@@ -70,6 +73,7 @@ def __init__(self, func: Func, param_spec: "Param") -> None:
             None,
             func.venv_backend,
             func.venv_params,
+            func.should_warn,
         )
         self.param_spec = param_spec
         self.session_signature = "({})".format(param_spec)

nox/_options.py

@@ -60,6 +60,49 @@ def _session_filters_merge_func(
     return getattr(command_args, key)
 
 
+def _default_venv_backend_merge_func(
+    command_args: argparse.Namespace, noxfile_args: argparse.Namespace
+) -> str:
+    """Merge default_venv_backend from command args and nox file. Default is "virtualenv".
+
+    Args:
+        command_args (_option_set.Namespace): The options specified on the
+            command-line.
+        noxfile_Args (_option_set.Namespace): The options specified in the
+            Noxfile.
+    """
+    return (
+        command_args.default_venv_backend
+        or noxfile_args.default_venv_backend
+        or "virtualenv"
+    )
+
+
+def _force_venv_backend_merge_func(
+    command_args: argparse.Namespace, noxfile_args: argparse.Namespace
+) -> str:
+    """Merge force_venv_backend from command args and nox file. Default is None.
+
+    Args:
+        command_args (_option_set.Namespace): The options specified on the
+            command-line.
+        noxfile_Args (_option_set.Namespace): The options specified in the
+            Noxfile.
+    """
+    if command_args.no_venv:
+        if (
+            command_args.force_venv_backend is not None
+            and command_args.force_venv_backend != "none"
+        ):
+            raise ValueError(
+                "You can not use `--no-venv` with a non-none `--force-venv-backend`"
+            )
+        else:
+            return "none"
+    else:
+        return command_args.force_venv_backend or noxfile_args.force_venv_backend
+
+
 def _envdir_merge_func(
     command_args: argparse.Namespace, noxfile_args: argparse.Namespace
 ) -> str:
@@ -221,6 +264,38 @@ def _session_completer(
         help="Logs the output of all commands run including commands marked silent.",
         noxfile=True,
     ),
+    _option_set.Option(
+        "default_venv_backend",
+        "-db",
+        "--default-venv-backend",
+        group=options.groups["secondary"],
+        noxfile=True,
+        merge_func=_default_venv_backend_merge_func,
+        help="Virtual environment backend to use by default for nox sessions, this is ``'virtualenv'`` by default but "
+        "any of ``('virtualenv', 'conda', 'venv')`` are accepted.",
+        choices=["none", "virtualenv", "conda", "venv"],
+    ),
+    _option_set.Option(
+        "force_venv_backend",
+        "-fb",
+        "--force-venv-backend",
+        group=options.groups["secondary"],
+        noxfile=True,
+        merge_func=_force_venv_backend_merge_func,
+        help="Virtual environment backend to force-use for all nox sessions in this run, overriding any other venv "
+        "backend declared in the nox file and ignoring the default backend. Any of ``('virtualenv', 'conda', 'venv')`` "
+        "are accepted.",
+        choices=["none", "virtualenv", "conda", "venv"],
+    ),
+    _option_set.Option(
+        "no_venv",
+        "--no-venv",
+        group=options.groups["secondary"],
+        default=False,
+        action="store_true",
+        help="Runs the selected sessions directly on the current interpreter, without creating a venv. This is an alias "
+        "for '--force-venv-backend none'.",
+    ),
     *_option_set.make_flag_pair(
         "reuse_existing_virtualenvs",
         ("-r", "--reuse-existing-virtualenvs"),

nox/manifest.py

@@ -21,6 +21,9 @@
 from nox.sessions import Session, SessionRunner
 
 
+WARN_PYTHONS_IGNORED = "python_ignored"
+
+
 class Manifest:
     """Session manifest.
 
@@ -170,6 +173,18 @@ def make_session(
         """
         sessions = []
 
+        # if backend is none we wont parametrize the pythons
+        backend = (
+            self._config.force_venv_backend
+            or func.venv_backend
+            or self._config.default_venv_backend
+        )
+        if backend == "none" and isinstance(func.python, (list, tuple, set)):
+            # we can not log a warning here since the session is maybe deselected.
+            # instead let's set a flag, to warn later when session is actually run.
+            func.should_warn[WARN_PYTHONS_IGNORED] = func.python
+            func.python = False
+
         # If the func has the python attribute set to a list, we'll need
         # to expand them.
         if isinstance(func.python, (list, tuple, set)):

nox/sessions.py

@@ -24,6 +24,7 @@
     Callable,
     Dict,
     Iterable,
+    Tuple,
     List,
     Mapping,
     Optional,
@@ -36,7 +37,7 @@
 from nox import _typing
 from nox._decorators import Func
 from nox.logger import logger
-from nox.virtualenv import CondaEnv, ProcessEnv, VirtualEnv
+from nox.virtualenv import CondaEnv, ProcessEnv, VirtualEnv, PassthroughEnv
 
 if _typing.TYPE_CHECKING:
     from nox.manifest import Manifest
@@ -279,25 +280,23 @@ def conda_install(self, *args: str, **kwargs: Any) -> None:
         .. _conda install:
         """
         venv = self._runner.venv
-        if not isinstance(venv, CondaEnv):
+
+        prefix_args = ()  # type: Tuple[str, ...]
+        if isinstance(venv, CondaEnv):
+            prefix_args = ("--prefix", venv.location)
+        elif not isinstance(venv, PassthroughEnv):  # pragma: no cover
             raise ValueError(
                 "A session without a conda environment can not install dependencies from conda."
             )
+
         if not args:
             raise ValueError("At least one argument required to install().")
 
         if "silent" not in kwargs:
             kwargs["silent"] = True
 
         self._run(
-            "conda",
-            "install",
-            "--yes",
-            "--prefix",
-            venv.location,
-            *args,
-            external="error",
-            **kwargs
+            "conda", "install", "--yes", *prefix_args, *args, external="error", **kwargs
         )
 
     def install(self, *args: str, **kwargs: Any) -> None:
@@ -325,7 +324,9 @@ def install(self, *args: str, **kwargs: Any) -> None:
 
         .. _pip: https://pip.readthedocs.org
         """
-        if not isinstance(self._runner.venv, (CondaEnv, VirtualEnv)):
+        if not isinstance(
+            self._runner.venv, (CondaEnv, VirtualEnv, PassthroughEnv)
+        ):  # pragma: no cover
             raise ValueError(
                 "A session without a virtualenv can not install dependencies."
             )
@@ -400,29 +401,35 @@ def envdir(self) -> str:
         return _normalize_path(self.global_config.envdir, self.friendly_name)
 
     def _create_venv(self) -> None:
-        if self.func.python is False:
-            self.venv = ProcessEnv()
+        backend = (
+            self.global_config.force_venv_backend
+            or self.func.venv_backend
+            or self.global_config.default_venv_backend
+        )
+
+        if backend == "none" or self.func.python is False:
+            self.venv = PassthroughEnv()
             return
 
         reuse_existing = (
             self.func.reuse_venv or self.global_config.reuse_existing_virtualenvs
         )
 
-        if not self.func.venv_backend or self.func.venv_backend == "virtualenv":
+        if backend is None or backend == "virtualenv":
             self.venv = VirtualEnv(
                 self.envdir,
                 interpreter=self.func.python,  # type: ignore
                 reuse_existing=reuse_existing,
                 venv_params=self.func.venv_params,
             )
-        elif self.func.venv_backend == "conda":
+        elif backend == "conda":
             self.venv = CondaEnv(
                 self.envdir,
                 interpreter=self.func.python,  # type: ignore
                 reuse_existing=reuse_existing,
                 venv_params=self.func.venv_params,
             )
-        elif self.func.venv_backend == "venv":
+        elif backend == "venv":
             self.venv = VirtualEnv(
                 self.envdir,
                 interpreter=self.func.python,  # type: ignore
@@ -433,7 +440,7 @@ def _create_venv(self) -> None:
         else:
             raise ValueError(
                 "Expected venv_backend one of ('virtualenv', 'conda', 'venv'), but got '{}'.".format(
-                    self.func.venv_backend
+                    backend
                 )
             )
 

nox/tasks.py

@@ -24,7 +24,7 @@
 from colorlog.escape_codes import parse_colors
 from nox import _options, registry
 from nox.logger import logger
-from nox.manifest import Manifest
+from nox.manifest import Manifest, WARN_PYTHONS_IGNORED
 from nox.sessions import Result
 
 
@@ -233,6 +233,14 @@ def run_manifest(manifest: Manifest, global_config: Namespace) -> List[Result]:
     # Note that it is possible for the manifest to be altered in any given
     # iteration.
     for session in manifest:
+        # possibly raise warnings associated with this session
+        if WARN_PYTHONS_IGNORED in session.func.should_warn:
+            logger.warning(
+                "Session {} is set to run with venv_backend='none', IGNORING its python={} parametrization. ".format(
+                    session.name, session.func.should_warn[WARN_PYTHONS_IGNORED]
+                )
+            )
+
         result = session.execute()
         result.log(
             "Session {name} {status}.".format(

nox/virtualenv.py

@@ -141,6 +141,16 @@ def _clean_location(self: "Union[CondaEnv, VirtualEnv]") -> bool:
     return True
 
 
+class PassthroughEnv(ProcessEnv):
+    """Represents the environment used to run nox itself
+
+    For now, this class is empty but it might contain tools to grasp some
+    hints about the actual env.
+    """
+
+    pass
+
+
 class CondaEnv(ProcessEnv):
     """Conda environment management class.
 

tests/resources/noxfile_pythons.py

@@ -0,0 +1,7 @@
+import nox
+
+
+@nox.session(python=["3.6"])
+@nox.parametrize("cheese", ["cheddar", "jack", "brie"])
+def snack(unused_session, cheese):
+    print("Noms, {} so good!".format(cheese))