Use PEP 440 style requirements

[cognifloyd]

This adds support for PEP 440 requirements instead of using the pip-specific VCS format.
see: https://peps.python.org/pep-0440/#direct-references

This facilitates using newer tooling that does not support using the legacy pip VCS format.

Summary of changes:

• *requirements.txt: reformat 6 requirements repeated across 11 files (this is the primary part of the PR, everything else supports it)
• Python: add 6 lines to 19 copies of dist_utils.py
• Python: add 8 lines and change 1 line in scripts/fixate-requirements.py

*requirements.txt

So, this change moves from this legacy pip VCS format:

st2/requirements.txt

Lines 23 to 25 in be437e9

	git+/~https://github.com/StackStorm/st2-auth-backend-flat-file.git@master#egg=st2-auth-backend-flat-file
	git+/~https://github.com/StackStorm/st2-auth-ldap.git@master#egg=st2-auth-ldap
	git+/~https://github.com/StackStorm/st2-rbac-backend.git@master#egg=st2-rbac-backend

to this PEP 440 direct reference format:

st2/requirements.txt

Lines 69 to 71 in 4d46a28

	st2-auth-backend-flat-file@ git+/~https://github.com/StackStorm/st2-auth-backend-flat-file.git@master
	st2-auth-ldap@ git+/~https://github.com/StackStorm/st2-auth-ldap.git@master
	st2-rbac-backend@ git+/~https://github.com/StackStorm/st2-rbac-backend.git@master

logshipper is one of the VCS requirements that we convert to PEP 440. It is linux-only and cannot be installed on MacOS X. So, this also adds platform_system=="Linux" to the logshipper dep (which was already done for logshipper's pyinotify dep) to allow building a virtualenv on MacOS X. That looks like this:

st2/requirements.txt

Line 31 in 4d46a28

logshipper@ git+/~https://github.com/StackStorm/logshipper.git@stackstorm_patched ; platform_system=="Linux"

And it standardizes the markers format (space before and after ;) of the pyinotify dep which is required by logshipper.

dist_utils.py

I adjusted the dist_utils.py functions to maintain the same functionality. Note however that I plan to use pants to replace all the python packaging bits, including dist_utils.py. So the dist_utils.py changes are a stop-gap to keep things working until we get the rest of the pants changes in. I added these lines to 20 copies of the dist_utils.py file:

st2/scripts/dist_utils.py

Lines 125 to 130 in 2b0cd75

	elif vcs_prefix in line and line.count("@") == 2:
	# PEP 440 direct reference: <package name>@ <url>@version
	req_name, link = line.split("@", 1)
	req_name = req_name.strip()
	link = f"{link.strip()}#egg={req_name}"
	return link, req_name

fixate-requirements.py

I also had to make a slight adjustment to fixate-requirements.py. This codepath is only used on CircleCI during package build when rpmbuild runs a make target that runs fixate-requirements. It does not run normally because we use pip 20, but rpmbuild runs the script before pipis updated from 19 to 20. So, this makes that corner case in our circleci workflow work like everywhere else we run fixate-requirements. Here is the new code in fixate-requirements.py:

st2/scripts/fixate-requirements.py

Lines 228 to 235 in 97dd34f

	if hasattr(req, "req") and req.req and str(req.req).count("@") == 2:
	# PEP 440 direct reference
	rline = str(linkreq.req)
	# Also write out environment markers
	if linkreq.markers:
	rline += " ; {}".format(str(linkreq.markers))
	else:
	rline = str(linkreq.link)

---

This depends on StackStorm/st2-packages#722

[rush-skills]

LGTM

[amanda11]

LGTM