Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow dockerfiles to be anywhere #178

Merged
merged 5 commits into from
Feb 8, 2023
Merged

allow dockerfiles to be anywhere #178

merged 5 commits into from
Feb 8, 2023

Conversation

jhsinger-klotho
Copy link
Contributor

• Does any part of it require special attention?
• Does it relate to or fix any issue? closes #74

we currently require dockerfiles to be at the klotho root path. With this change if a custom dockerfile is provided we will obey the location it is provided at in regards to the root path of the klotho invocation.

In order to make this work we have to add any annotated file as a sourcefile to prevent it from getting pruned /~https://github.com/klothoplatform/klotho-pro/pull/50 is the pro request for the same.

to standardize this logic in pulumi, i created a createImage function. We then store the exec unit images in a map to be retrieved by the other functions so we dont have to recreate this in 4 places and muddy the params to those other functions

Standard checks

  • Unit tests: Any special considerations?
  • Docs: Do we need to update any docs, internal or public? will remove the note in docs when this goes in
  • Backwards compatibility: Will this break existing apps? If so, what would be the extra work required to keep them working? yes

Copy link
Contributor

@gordon-klotho gordon-klotho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only blocking is the createImage comment.

@@ -12,8 +12,10 @@ import (

var multilineCommentMarginRegexp = regexp.MustCompile(`(?m)^\s*[*]*[ \t]*`) // we need to use [ \t] instead of \s, because \s includes newlines in (?m) mode.

const js = core.LanguageId("javascript")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this into the struct a while back and just use Language.ID (since it's exported and IMO clearer what it is).

const image = this.sharedRepo.buildAndPushImage({
context: `./${execUnitName}`,
context: `./${execUnitName}/${dockerfilePath}`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be

{
    context: `./${execUnitName}/${path.dirname(dockerfilePath)}`,
    dockerfile: `./${execUnitName}/${dockerfilePath}`,
}

Though I'm not sure if using that context is correct versus using ./${execUnitName}.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so i printed out what my code has for the context and its ./main/src which would be correct

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can leave context at the root of the exec unit if thats what you mean

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the path is already filepath.Dir in go so it doesnt include the file name

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, isn't that wrong then? What if your dockerfile was build/myservice.Dockerfile? At the least the field name doesn't match its content.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i might be a bit confused by what you mean. Doesnt it assume that there is 1 and only 1 dockerfile within the context and it will use that? if it was build/myservice.Dockerfile then we would point to {execunit}/build/myservice.Dockerfile which i would think is correct

Copy link
Contributor

@gordon-klotho gordon-klotho Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file parameter defaults to Dockerfile, but I don't believe it's relative to the context but rather the current working directory from when it's run (see below). I don't believe it does any sort of searching. Additionally, by sending the context of {execunit}/build that would mean it couldn't include any source files in {execunit}/src (or similar, outside of the build directory).

nm on the first (default), part:

Usage:  docker build [OPTIONS] PATH | URL | -
[...]
  -f, --file string             Name of the Dockerfile (Default is 'PATH/Dockerfile')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

theres no way we can understand the context without parsing the Dockerfile, so are you suggesting just leaving it at the root exec unit so we can encompass everything? If so then thats fine with me and similarly is the question, does the --file parameter only look for files named Dockerfile or would it find any Dockerfile?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

theres no way we can understand the context without parsing the Dockerfile

Not only that, but even that might not be able to tell you enough (though in many circumstances we could make a guess based on any ADD/COPY statements). This would have to be a directive to support all the different edge cases.

are you suggesting just leaving it at the root exec unit so we can encompass everything?

Yeah, for now until we get some more intelligence built in.

does the --file parameter only look for files named Dockerfile or would it find any Dockerfile?

I'm relatively certain it only finds the file named Dockerfile. Most non-trivial build processes I've seen use explicit -f path/to/dockerfile for this reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok i can go make these updates then, will have a new revision out today

@github-actions
Copy link

github-actions bot commented Feb 6, 2023

Package Line Rate Health
github.com/klothoplatform/klotho/pkg/analytics 2%
github.com/klothoplatform/klotho/pkg/annotation 23%
github.com/klothoplatform/klotho/pkg/cli 4%
github.com/klothoplatform/klotho/pkg/core 21%
github.com/klothoplatform/klotho/pkg/env_var 82%
github.com/klothoplatform/klotho/pkg/exec_unit 53%
github.com/klothoplatform/klotho/pkg/infra/kubernetes 59%
github.com/klothoplatform/klotho/pkg/infra/kubernetes/helm 39%
github.com/klothoplatform/klotho/pkg/input 63%
github.com/klothoplatform/klotho/pkg/lang 38%
github.com/klothoplatform/klotho/pkg/lang/dockerfile 0%
github.com/klothoplatform/klotho/pkg/lang/golang 19%
github.com/klothoplatform/klotho/pkg/lang/javascript 47%
github.com/klothoplatform/klotho/pkg/lang/python 60%
github.com/klothoplatform/klotho/pkg/lang/yaml 0%
github.com/klothoplatform/klotho/pkg/logging 7%
github.com/klothoplatform/klotho/pkg/multierr 95%
github.com/klothoplatform/klotho/pkg/provider/aws 59%
github.com/klothoplatform/klotho/pkg/runtime 78%
github.com/klothoplatform/klotho/pkg/static_unit 33%
github.com/klothoplatform/klotho/pkg/validation 73%
github.com/klothoplatform/klotho/pkg/yaml_util 79%
Summary 40% (3796 / 9457)

@jhsinger-klotho
Copy link
Contributor Author

the branch got a bit messed up, so i had to rebase to fix conflicts with main so the changes from the last commit should just be to address the comments, but hard to see now

Copy link
Contributor

@gordon-klotho gordon-klotho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the js change in parser.go is used for or why the change, but the important bits (things related to Dockerfile) are good.

@jhsinger-klotho jhsinger-klotho merged commit bb5f5fe into main Feb 8, 2023
@jhsinger-klotho jhsinger-klotho deleted the dockerfile branch February 8, 2023 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow dockerfiles not at the root of klotho dir
3 participants