Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make fileref use root path and stat unit have same logic as embed #34

Merged
merged 2 commits into from
Jan 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions pkg/core/file_ref.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@ package core
import (
"io"
"os"
"path/filepath"
)

// FileRef is a lightweight representation of a file, deferring reading its contents until `WriteTo` is called.
type FileRef struct {
FPath string
FPath string
RootConfigPath string
}

func (r *FileRef) Clone() File {
Expand All @@ -19,7 +21,7 @@ func (r *FileRef) Path() string {
}

func (r *FileRef) WriteTo(w io.Writer) (int64, error) {
f, err := os.Open(r.FPath)
f, err := os.Open(filepath.Join(r.RootConfigPath, r.FPath))
if err != nil {
return 0, err
}
Expand Down
5 changes: 4 additions & 1 deletion pkg/input/readdir.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,10 @@ func ReadDir(fsys fs.FS, cfg config.Application, cfgFilePath string) (*core.Inpu
// (so './dist/index.js' becomes 'index.js')
relPath = info.Name()
}
var f core.File = &core.FileRef{FPath: relPath}
var f core.File = &core.FileRef{
FPath: relPath,
RootConfigPath: cfg.Path,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to be mindful of here is that this path is relative to the "root", which is split from the original passed-in path in splitPathRoot. This is to prevent plugins from accessing arbitrary files (eg, FPath: "../../../etc/passwd").

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when do we allow access to files outside of the root of the klotho run? Im failing to understand this a little bit so do you have a scenario?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was more a defense against mistakes and third-party plugins (when we get them). Might not be necessary for now, but it does mean this could be incorrect - try doing klotho ../ --app blah

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, let me make a note of that and run through that scenario, if it doesnt work, ill cut a card to fix it

}

if info.IsDir() {
switch info.Name() {
Expand Down
52 changes: 50 additions & 2 deletions pkg/static_unit/plugin_static_unit.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ package staticunit

import (
"errors"
"path/filepath"
"strconv"
"strings"

"github.com/bmatcuk/doublestar/v4"
"github.com/klothoplatform/klotho/pkg/config"
Expand Down Expand Up @@ -63,11 +65,16 @@ func (p StaticUnitSplit) Transform(result *core.CompilationResult, deps *core.De
matcher := staticAssetPathMatcher{}
matcher.staticFiles, _ = cap.Directives.StringArray("static_files")
matcher.sharedFiles, _ = cap.Directives.StringArray("shared_files")

err := matcher.ModifyPathsForAnnotatedFile(f.Path())
if err != nil {
errs.Append(err)
break
}
matchCount := 0
for _, asset := range input.Files() {
ref := &core.FileRef{
FPath: asset.Path(),
FPath: asset.Path(),
RootConfigPath: p.Config.Path,
}
static, shared := matcher.Matches(asset.Path())
if shared {
Expand Down Expand Up @@ -98,6 +105,47 @@ type staticAssetPathMatcher struct {
err error
}

// ModifyPathsForAnnotatedFile transforms the staticAssetPathMatcher sharedFiles and staticFiles to be absolute paths from the klotho project root, without the prefix '/'.
// This is done to conform to the file path structure of input files.
func (m *staticAssetPathMatcher) ModifyPathsForAnnotatedFile(path string) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you doc a bit what this function does? I'm getting lost in the "Abs", "Rel" etc. It likely is similar to how readdir works, but I've forgotten how that works 😄

newInclude := []string{}
for _, pattern := range m.staticFiles {
absPath, err := filepath.Abs(pattern)
if err != nil {
return err
}
if absPath == pattern {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not quite understanding why this is a special case. Won't every absPath have a /? Why is the prefix only trimmed if the filpath.Abs(pattern) seemingly does nothing to the path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so our paths for files are like src/somefile.txt

if they specify an absolute path of /src/somefile.txt we wont match unless we remove the prefix /

The other scenario after that is for relative paths, which wont start with / anyways.

The only real reason we check absPath == pattern is just to make sure its a true absolute path.

If that doesnt make sense we can rework this or have another discussion, going to push to get the search the deck stuff going for ala

newInclude = append(newInclude, strings.TrimPrefix(pattern, "/"))
continue
}
relPath, err := filepath.Rel(filepath.Dir("."), filepath.Join(filepath.Dir(path), pattern))
if err != nil {
return err
}
newInclude = append(newInclude, relPath)
}
m.staticFiles = newInclude

newExclude := []string{}
for _, pattern := range m.sharedFiles {
absPath, err := filepath.Abs(pattern)
if err != nil {
return err
}
if absPath == pattern {
newExclude = append(newExclude, strings.TrimPrefix(pattern, "/"))
continue
}
relPath, err := filepath.Rel(filepath.Dir("."), filepath.Join(filepath.Dir(path), pattern))
if err != nil {
return err
}
newExclude = append(newExclude, relPath)
}
m.sharedFiles = newExclude
return nil
}

func (m *staticAssetPathMatcher) Matches(p string) (bool, bool) {
if m.err != nil {
return false, false
Expand Down
74 changes: 74 additions & 0 deletions pkg/static_unit/plugin_static_unit_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,77 @@ func Test_assetPathMatcher_Matches(t *testing.T) {
})
}
}

func Test_assetPathMatcher_ModifyPathsForAnnotatedFile(t *testing.T) {
type testResult struct {
include []string
exclude []string
}
tests := []struct {
name string
matcher staticAssetPathMatcher
path string
want testResult
}{
{
name: "simple relative match",
matcher: staticAssetPathMatcher{staticFiles: []string{"file.txt"}, sharedFiles: []string{"notfile.txt"}},
path: "file.txt",
want: testResult{include: []string{"file.txt"}, exclude: []string{"notfile.txt"}},
},
{
name: "nested relative match",
matcher: staticAssetPathMatcher{staticFiles: []string{"file.txt"}, sharedFiles: []string{"notfile.txt"}},
path: "dir/file.txt",
want: testResult{include: []string{"dir/file.txt"}, exclude: []string{"dir/notfile.txt"}},
},
{
name: "simple absolute match",
matcher: staticAssetPathMatcher{staticFiles: []string{"/file.txt"}, sharedFiles: []string{"/notfile.txt"}},
path: "file.txt",
want: testResult{include: []string{"file.txt"}, exclude: []string{"notfile.txt"}},
},
{
name: "nested absolute match",
matcher: staticAssetPathMatcher{staticFiles: []string{"/dir/file.txt"}, sharedFiles: []string{"/dir/notfile.txt"}},
path: "dir/file.txt",
want: testResult{include: []string{"dir/file.txt"}, exclude: []string{"dir/notfile.txt"}},
},
{
name: "mix relative and absolute match",
matcher: staticAssetPathMatcher{staticFiles: []string{"/dir/file.txt", "other.txt"}, sharedFiles: []string{"/dir/notfile.txt", "notother.txt"}},
path: "dir/file.txt",
want: testResult{include: []string{"dir/file.txt", "dir/other.txt"}, exclude: []string{"dir/notfile.txt", "dir/notother.txt"}},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert := assert.New(t)

err := tt.matcher.ModifyPathsForAnnotatedFile(tt.path)
if !assert.NoError(err) {
return
}

for _, wantPath := range tt.want.include {
found := false
for _, path := range tt.matcher.staticFiles {
if wantPath == path {
found = true
}
}
assert.True(found)
}

for _, wantPath := range tt.want.exclude {
found := false
for _, path := range tt.matcher.sharedFiles {
if wantPath == path {
found = true
}
}
assert.True(found)
}
})
}
}