Chef Export no longer includes files outside of CookbookVersionLoader segments such as test/ or spec/

[jayhendren]

ChefDK::PolicyfileServices::ExportRepo calls out to Chef::Cookbook::CookbookVersionLoader, which hardcodes a whitelist of files and folders, in order to determine what files to upload to a testkitchen box. This prevents other files and folders in a cookbook from being uploaded. Other kitchen provisioners provided by testkitchen use the cookbook_files_glob kitchen configuration option to control what files will be uploaded, but the policyfile_zero provisioner doesn't even look at this config option, instead relying completely on CookbookVersionLoader to figure out what to upload.

[danielsdeleo]

What kind of other files do you need? It's more likely that we'll solve this at the CookbookVersionLoader level which would make this technically a Chef issue but we can track it here as well.

[jayhendren]

In our particular case, our cookbooks expose integration tests so that those tests can be run by downstream cookbooks. This is particularly helpful for policy-level acceptance & integration testing. See test-kitchen/test-kitchen#843.

Though I could imagine a use-case for any non-standard cookbook repo layout, for whatever reason.

I would also argue that this issue should be categorized as a "bug" rather than "enhancement" - in the previous release of ChefDK (0.10.0), the policyfile_zero provisioner simply uploaded all files in a cookbook, which allowed for the workflow I've described, and this workflow is now broken in ChefDK 0.11.2.

[danielsdeleo]

The plan is to fix this via chef/chef#4629 The end result would be that everything that isn't either explicitly ignored or contained in a dotdir at the cookbook root would be exported (see the pull request and code comments therein for more about that).

If you specifically want cookbook_files_glob option, I'd consider that an enhancement, but the other way should get you back the behavior you had before (while still respecting chefignore, which is why the change was made in the first place).

[jayhendren]

I see what you're saying now - yes, there's a difference between adding support for cookbook_files_glob and the fixing problems introduced by switching to CookbookVersionLoader. The latter problem is what I'm more interested in and I had assumed it was more closely connected with the lack of support for the cookbook_files_glob feature than was actually the case. It looks like the solution presented in the referenced PR will make cookbook_files_glob unnecessary anyway. Thanks for looking into this and helping to solve!

[danielsdeleo]

I put up a PR for this here: #716

Will be on vacation next week, but I will look into it after if no one finishes that up.