Skip to content

Commit

Permalink
msys2-runtime: support OneDrive better
Browse files Browse the repository at this point in the history 
With this change, files are no longer "hydrated" (read: downloaded on
the fly) when calling `ls` _just_ to determine whether the files in
question have a `#!` line that makes them executable.

This commit brings the changes from
msys2/msys2-runtime#209 to MSYS2-packages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
  • Loading branch information
@dscho
dscho committed Mar 23, 2024
1 parent 302a39c commit 41fbcb0
Show file tree
Hide file tree
Showing 5 changed files with 450 additions and 7 deletions.
25 changes: 25 additions & 0 deletions msys2-runtime/0050-pathconv-don-t-skip-arguments-with-double-quote.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
From 65672ed072d895361382478d82b33f8c11c7201d Mon Sep 17 00:00:00 2001
From: Christoph Reiter <reiter.christoph@gmail.com>
Date: Sat, 10 Feb 2024 08:31:55 +0100
Subject: [PATCH 50/N] pathconv: don't skip arguments with double quote

It is used to pass strings/paths to the preprocessor and breaks
for example the CPython build. For example -DPREFIX='"/ucrt64"'.

Fixes #190
---
winsup/cygwin/msys2_path_conv.cc | 1 -
1 file changed, 1 deletion(-)

diff --git a/winsup/cygwin/msys2_path_conv.cc b/winsup/cygwin/msys2_path_conv.cc
index f6e91a2..4c0cc82 100644
--- a/winsup/cygwin/msys2_path_conv.cc
+++ b/winsup/cygwin/msys2_path_conv.cc
@@ -368,7 +368,6 @@ skip_p2w:
switch (*it) {
case '`':
case '\'':
- case '"':
case '*':
case '?':
case '[':
83 changes: 83 additions & 0 deletions msys2-runtime/0051-Work-around-fragile-include-in-binutils.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
From 44b5e4eed0f2f119d312fe718c2124250c8626f9 Mon Sep 17 00:00:00 2001
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Fri, 2 Feb 2024 13:40:28 +0100
Subject: [PATCH 51/N] Work around fragile `#include` in binutils
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The `bfd.h` header file that is included in `binutils` has the line
`#include "ansidecl.h"`, which is fragile because it prefers Cygwin's
`include/ansidecl.h` (as opposed to `#include <ansidecl.h>`, which would
only look in the system include paths).

This matters because as of v2.42, `bfd.h` also makes use of the
`ATTRIBUTE_WARN_UNUSED_RESULT` macro.

So let's just copy that macro (and while at it, the other `ATTRIBUTE_*`
macros) from binutils' `ansidecl.h` file, to avoid compile errors while
compiling `dumper.o` that look like this:

/usr/include/bfd.h:2770:1: error: expected initializer before ‘ATTRIBUTE_WARN_UNUSED_RESULT’
2770 | ATTRIBUTE_WARN_UNUSED_RESULT;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
include/ansidecl.h | 43 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)

diff --git a/include/ansidecl.h b/include/ansidecl.h
index 6e4bfc2..ceb356e 100644
--- a/include/ansidecl.h
+++ b/include/ansidecl.h
@@ -283,6 +283,49 @@ So instead we use the macro below and test it against specific values. */
# endif /* GNUC >= 4.9 */
#endif /* ATTRIBUTE_NO_SANITIZE_UNDEFINED */

+/* Attribute 'nonstring' was valid as of gcc 8. */
+#ifndef ATTRIBUTE_NONSTRING
+# if GCC_VERSION >= 8000
+# define ATTRIBUTE_NONSTRING __attribute__ ((__nonstring__))
+# else
+# define ATTRIBUTE_NONSTRING
+# endif
+#endif
+
+/* Attribute `alloc_size' was valid as of gcc 4.3. */
+#ifndef ATTRIBUTE_RESULT_SIZE_1
+# if (GCC_VERSION >= 4003)
+# define ATTRIBUTE_RESULT_SIZE_1 __attribute__ ((alloc_size (1)))
+# else
+# define ATTRIBUTE_RESULT_SIZE_1
+#endif
+#endif
+
+#ifndef ATTRIBUTE_RESULT_SIZE_2
+# if (GCC_VERSION >= 4003)
+# define ATTRIBUTE_RESULT_SIZE_2 __attribute__ ((alloc_size (2)))
+# else
+# define ATTRIBUTE_RESULT_SIZE_2
+#endif
+#endif
+
+#ifndef ATTRIBUTE_RESULT_SIZE_1_2
+# if (GCC_VERSION >= 4003)
+# define ATTRIBUTE_RESULT_SIZE_1_2 __attribute__ ((alloc_size (1, 2)))
+# else
+# define ATTRIBUTE_RESULT_SIZE_1_2
+#endif
+#endif
+
+/* Attribute `warn_unused_result' was valid as of gcc 3.3. */
+#ifndef ATTRIBUTE_WARN_UNUSED_RESULT
+# if GCC_VERSION >= 3003
+# define ATTRIBUTE_WARN_UNUSED_RESULT __attribute__ ((__warn_unused_result__))
+# else
+# define ATTRIBUTE_WARN_UNUSED_RESULT
+# endif
+#endif
+
/* We use __extension__ in some places to suppress -pedantic warnings
about GCC extensions. This feature didn't work properly before
gcc 2.8. */
278 changes: 278 additions & 0 deletions msys2-runtime/0052-Cygwin-try-to-avoid-recalling-offline-files.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
From 40793722b8ea98fcdf63533e10ebd28e84b7ffaa Mon Sep 17 00:00:00 2001
From: Corinna Vinschen <corinna@vinschen.de>
Date: Fri, 8 Mar 2024 20:57:06 +0100
Subject: [PATCH 52/N] Cygwin: try to avoid recalling offline files

Chances are high that Cygwin recalls offline files from remote
storage, even if the file is only accessed during stat(2) or
readdir(3).

To avoid this
- make sure Cygwin is placeholder-aware,
- open files in path_conv handling, as well as in stat(2)/readdir(3)
scenarios with FILE_OPEN_NO_RECALL, and
- during symlink checking or testing for executablility, don't even
try to open the file if one of the OFFLINE attributes is set.

Reported-by: Marcin Wisnicki <mwisnicki@gmail.com>
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
winsup/cygwin/autoload.cc | 1 +
winsup/cygwin/dcrt0.cc | 3 +++
winsup/cygwin/fhandler/disk_file.cc | 20 ++++++++++++++------
winsup/cygwin/local_includes/ntdll.h | 8 ++++++++
winsup/cygwin/local_includes/path.h | 16 +++++++++++++++-
winsup/cygwin/local_includes/winlean.h | 8 ++++++++
winsup/cygwin/path.cc | 17 +++++++++++++----
7 files changed, 62 insertions(+), 11 deletions(-)

diff --git a/winsup/cygwin/autoload.cc b/winsup/cygwin/autoload.cc
index cdf6e75..c579bef 100644
--- a/winsup/cygwin/autoload.cc
+++ b/winsup/cygwin/autoload.cc
@@ -477,6 +477,7 @@ LoadDLLfuncEx (SetThreadDescription, 8, KernelBase, 1)
LoadDLLfunc (VirtualAlloc2, 28, KernelBase)

LoadDLLfunc (NtMapViewOfSectionEx, 36, ntdll)
+LoadDLLfuncEx (RtlSetProcessPlaceholderCompatibilityMode, 4, ntdll, 1)

LoadDLLfunc (ldap_bind_s, 0, wldap32)
LoadDLLfunc (ldap_count_entries, 0, wldap32)
diff --git a/winsup/cygwin/dcrt0.cc b/winsup/cygwin/dcrt0.cc
index d7a1b1a..a5b82b0 100644
--- a/winsup/cygwin/dcrt0.cc
+++ b/winsup/cygwin/dcrt0.cc
@@ -821,6 +821,9 @@ dll_crt0_1 (void *)
if (dynamically_loaded)
sigproc_init ();

+ /* Call this before accessing any files. */
+ RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);
+
check_sanity_and_sync (user_data);

/* Initialize malloc and then call user_shared_initialize since it relies
diff --git a/winsup/cygwin/fhandler/disk_file.cc b/winsup/cygwin/fhandler/disk_file.cc
index 8528f7f..87c14df 100644
--- a/winsup/cygwin/fhandler/disk_file.cc
+++ b/winsup/cygwin/fhandler/disk_file.cc
@@ -175,7 +175,9 @@ readdir_check_reparse_point (POBJECT_ATTRIBUTES attr, bool remote)
bool ret = false;

status = NtOpenFile (&reph, READ_CONTROL, attr, &io, FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT | FILE_OPEN_REPARSE_POINT);
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT
+ | FILE_OPEN_REPARSE_POINT);
if (NT_SUCCESS (status))
{
PREPARSE_DATA_BUFFER rp = (PREPARSE_DATA_BUFFER) tp.c_get ();
@@ -326,6 +328,7 @@ fhandler_base::fstat_by_name (struct stat *buf)
status = NtOpenFile (&dir, SYNCHRONIZE | FILE_LIST_DIRECTORY,
&attr, &io, FILE_SHARE_VALID_FLAGS,
FILE_SYNCHRONOUS_IO_NONALERT
+ | FILE_OPEN_NO_RECALL
| FILE_OPEN_FOR_BACKUP_INTENT
| FILE_DIRECTORY_FILE);
if (!NT_SUCCESS (status))
@@ -609,7 +612,8 @@ fhandler_disk_file::fstatvfs (struct statvfs *sfs)
opened = NT_SUCCESS (NtOpenFile (&fh, READ_CONTROL,
pc.get_object_attr (attr, sec_none_nih),
&io, FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT));
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT));
if (!opened)
{
/* Can't open file. Try again with parent dir. */
@@ -618,7 +622,8 @@ fhandler_disk_file::fstatvfs (struct statvfs *sfs)
attr.ObjectName = &dirname;
opened = NT_SUCCESS (NtOpenFile (&fh, READ_CONTROL, &attr, &io,
FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT));
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT));
if (!opened)
goto out;
}
@@ -2054,7 +2059,8 @@ readdir_get_ino (const char *path, bool dot_dot)
|| NT_SUCCESS (NtOpenFile (&hdl, READ_CONTROL,
pc.get_object_attr (attr, sec_none_nih),
&io, FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT
| (pc.is_known_reparse_point ()
? FILE_OPEN_REPARSE_POINT : 0)))
)
@@ -2103,8 +2109,9 @@ fhandler_disk_file::readdir_helper (DIR *dir, dirent *de, DWORD w32_err,
Mountpoints and unknown or unhandled reparse points will be treated
as normal file/directory/unknown. In all cases, returning the INO of
the reparse point (not of the target) matches behavior of posix systems.
+ Unless the file is OFFLINE. *.
*/
- if (attr & FILE_ATTRIBUTE_REPARSE_POINT)
+ if ((attr & FILE_ATTRIBUTE_REPARSE_POINT) && !isoffline (attr))
{
OBJECT_ATTRIBUTES oattr;

@@ -2345,7 +2352,8 @@ go_ahead:
&nfs_aol_ffei, sizeof nfs_aol_ffei)
: NtOpenFile (&hdl, READ_CONTROL, &attr, &io,
FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT
| FILE_OPEN_REPARSE_POINT);
if (NT_SUCCESS (f_status))
{
diff --git a/winsup/cygwin/local_includes/ntdll.h b/winsup/cygwin/local_includes/ntdll.h
index a4e8b88..aa1f3b9 100644
--- a/winsup/cygwin/local_includes/ntdll.h
+++ b/winsup/cygwin/local_includes/ntdll.h
@@ -159,6 +159,13 @@ extern GUID __cygwin_socket_guid;
#define FILE_VC_QUOTAS_REBUILDING 0x00000200
#define FILE_VC_VALID_MASK 0x000003ff

+#define PHCM_APPLICATION_DEFAULT 0
+#define PHCM_DISGUISE_PLACEHOLDER 1
+#define PHCM_EXPOSE_PLACEHOLDERS 2
+#define PHCM_MAX 2
+#define PHCM_ERROR_INVALID_PARAMETER -1
+#define PHCM_ERROR_NO_TEB -2
+
/* IOCTL code to impersonate client of named pipe. */

#define FSCTL_PIPE_DISCONNECT CTL_CODE(FILE_DEVICE_NAMED_PIPE, 1, \
@@ -1605,6 +1612,7 @@ extern "C"
BOOLEAN);
NTSTATUS RtlSetGroupSecurityDescriptor (PSECURITY_DESCRIPTOR, PSID, BOOLEAN);
NTSTATUS RtlSetOwnerSecurityDescriptor (PSECURITY_DESCRIPTOR, PSID, BOOLEAN);
+ CHAR RtlSetProcessPlaceholderCompatibilityMode (CHAR);
PUCHAR RtlSubAuthorityCountSid (PSID);
PULONG RtlSubAuthoritySid (PSID, ULONG);
ULONG RtlUnicodeStringToAnsiSize (PUNICODE_STRING);
diff --git a/winsup/cygwin/local_includes/path.h b/winsup/cygwin/local_includes/path.h
index 74f831e..8c97c42 100644
--- a/winsup/cygwin/local_includes/path.h
+++ b/winsup/cygwin/local_includes/path.h
@@ -23,6 +23,14 @@ has_attribute (DWORD attributes, DWORD attribs_to_test)
&& (attributes & attribs_to_test);
}

+extern inline bool
+isoffline (DWORD attributes)
+{
+ return has_attribute (attributes, FILE_ATTRIBUTE_OFFLINE
+ | FILE_ATTRIBUTE_RECALL_ON_OPEN
+ | FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS);
+}
+
enum executable_states
{
is_executable,
@@ -230,6 +238,12 @@ class path_conv
bool exists () const {return fileattr != INVALID_FILE_ATTRIBUTES;}
bool has_attribute (DWORD x) const {return exists () && (fileattr & x);}
int isdir () const {return has_attribute (FILE_ATTRIBUTE_DIRECTORY);}
+ bool isoffline () const
+ {
+ return has_attribute (FILE_ATTRIBUTE_OFFLINE
+ | FILE_ATTRIBUTE_RECALL_ON_OPEN
+ | FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS);
+ }
executable_states exec_state ()
{
extern int _check_for_executable;
@@ -237,7 +251,7 @@ class path_conv
return is_executable;
if (mount_flags & MOUNT_NOTEXEC)
return not_executable;
- if (!_check_for_executable)
+ if (isoffline () || !_check_for_executable)
return dont_care_if_executable;
return dont_know_if_executable;
}
diff --git a/winsup/cygwin/local_includes/winlean.h b/winsup/cygwin/local_includes/winlean.h
index 9b30b65..d4b4038 100644
--- a/winsup/cygwin/local_includes/winlean.h
+++ b/winsup/cygwin/local_includes/winlean.h
@@ -74,6 +74,14 @@ details. */
#undef CRITICAL
#endif

+/* Filesystem flags not yet supported by Mingw-w64 headers. */
+#ifndef FILE_ATTRIBUTE_RECALL_ON_OPEN
+#define FILE_ATTRIBUTE_RECALL_ON_OPEN 0x00040000
+#endif
+#ifndef FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS
+#define FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS 0x00400000
+#endif
+
/* So-called "Microsoft Account" SIDs (S-1-11-...) have a netbios domain name
"MicrosoftAccounts". The new "Application Container SIDs" (S-1-15-...)
have a netbios domain name "APPLICATION PACKAGE AUTHORITY"
diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
index af88ecf..e61d370 100644
--- a/winsup/cygwin/path.cc
+++ b/winsup/cygwin/path.cc
@@ -577,6 +577,7 @@ getfileattr (const char *path, bool caseinsensitive) /* path has to be always ab
status = NtOpenFile (&dir, SYNCHRONIZE | FILE_LIST_DIRECTORY,
&attr, &io, FILE_SHARE_VALID_FLAGS,
FILE_SYNCHRONOUS_IO_NONALERT
+ | FILE_OPEN_NO_RECALL
| FILE_OPEN_FOR_BACKUP_INTENT
| FILE_DIRECTORY_FILE);
if (NT_SUCCESS (status))
@@ -3353,7 +3354,8 @@ restart:
}
status = NtOpenFile (&h, READ_CONTROL | FILE_READ_ATTRIBUTES,
&attr, &io, FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_REPARSE_POINT
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_REPARSE_POINT
| FILE_OPEN_FOR_BACKUP_INTENT);
debug_printf ("%y = NtOpenFile (no-EAs %S)", status, &upath);
}
@@ -3481,6 +3483,7 @@ restart:
status = NtOpenFile (&dir, SYNCHRONIZE | FILE_LIST_DIRECTORY,
&dattr, &io, FILE_SHARE_VALID_FLAGS,
FILE_SYNCHRONOUS_IO_NONALERT
+ | FILE_OPEN_NO_RECALL
| FILE_OPEN_FOR_BACKUP_INTENT
| FILE_DIRECTORY_FILE);
if (!NT_SUCCESS (status))
@@ -3570,7 +3573,11 @@ restart:
directory using a relative path, symlink evaluation goes totally
awry. We never want a virtual drive evaluated as symlink. */
if (upath.Length <= 14)
- goto file_not_symlink;
+ goto file_not_symlink;
+
+ /* Offline files, even if reparse points, are not symlinks. */
+ if (isoffline (fileattr))
+ goto file_not_symlink;

/* Reparse points are potentially symlinks. This check must be
performed before checking the SYSTEM attribute for sysfile
@@ -3616,7 +3623,8 @@ restart:

status = NtOpenFile (&sym_h, SYNCHRONIZE | GENERIC_READ, &attr, &io,
FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT
| FILE_SYNCHRONOUS_IO_NONALERT);
if (!NT_SUCCESS (status))
res = 0;
@@ -3660,7 +3668,8 @@ restart:

status = NtOpenFile (&sym_h, SYNCHRONIZE | GENERIC_READ, &attr, &io,
FILE_SHARE_VALID_FLAGS,
- FILE_OPEN_FOR_BACKUP_INTENT
+ FILE_OPEN_NO_RECALL
+ | FILE_OPEN_FOR_BACKUP_INTENT
| FILE_SYNCHRONOUS_IO_NONALERT);

if (!NT_SUCCESS (status))
Loading

0 comments on commit 41fbcb0

Please sign in to comment.