Can't export non-resource files in directory with .gdignore

[charliewhitfield]

Godot version:

Does not work in v3.2.4-rc4 or -rc5.
Works in v3.2.4-rc3, v3.2.3, and earlier.

OS/device including version:

Using Windows 10 to export either Windows or HTML5.

Issue description:

Due to #38957, it is necessary to use .gdignore in directories with .csv files used for data. We use this in @ivoyager to read .csv files containing many thousands of astronomical data points. In v3.2.4-rc3, v3.2.3 and earlier, it is possible to export these projects. In v3.2.4-rc4 and -rc5, exports fail because the directories cannot be opened.

Steps to reproduce:
Create project with test_directory containing .gdignore and test.csv. Add "*.csv" to Export/Resources/Filters to export non-resource files/folders. Add code:

var dir_path := "res://test_directory"

func _ready():
	var dir := Directory.new()
	if dir.open(dir_path) != OK:
		print("Could not open dir: ", dir_path)
		return
	dir.list_dir_begin()
	var file_name := dir.get_next()
	while file_name:
		if !dir.current_is_dir():
			print(file_name)
		file_name = dir.get_next()

This will correctly print "test.csv" in editor mode (all versions) and in exports from v3.2.4-rc3*, v3.2.3, and earlier. But in v3.2.4-rc4 and -rc5, both Windows and HTML5 export will print "Could not open dir: res://test_directory".

(*Note: -rc3 had an unrelated issue that breaks HTML5 export; however, Windows export works as stated above.)

Minimal reproduction project:

test-rc5.zip

[akien-mga]

It's expected that files in a folder with .gdignore are neither imported nor exported, that's what .gdignore was intended for.

#38957 needs to be fixed properly to solve this, you shouldn't have to .gdignore your CSV files to be able to use them.

[charliewhitfield]

As a general solution, I wonder if we should have Import As: Do Not Import.

I assume that Import As: CSV is there for some future implementation that does not exist yet. However, I can imagine that there might be other situations where the developer wants to evade the import system, but still have files available for export.

If anyone thinks this is not a totally dumb idea, let me know and I'll open it as a feature proposal.

[akien-mga]

Yes we've been discussing this with core developers yesterday, and that was indeed the consensus:

1. Add a "Keep" import option which disables import for a given resource (helps with CSV, but also with all other resources where one might want to exclude some of them without relying on .gdignore in a subfolder).
2. Add a proper importer for CSV databases as a resource, which allows editing and retrieving cell values as one could do in a spreadsheet editor.

The 1st part should have priority as it provides an easy fix for #38957 and this issue. Hopefully we can get that implemented ASAP for 3.3. The 2nd part will need writing a proposal to discuss further whether it's considered useful, what the API should be like, etc.

The current "CSV" importer was just a very bad hack to prevent errors trying to parse CSV files as translations, but as #38957 shows it's possibly worse than the original issue.

[akien-mga]

Fixed by #47268 (and #47300 for 3.3).