Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration test with mock LLM #20

Merged
merged 16 commits into from
Jun 10, 2024
Merged

Conversation

tpaulshippy
Copy link
Contributor

Purpose

Automated testing the CLI (#2)

Approach

Tried a number of approaches, but landed on these:

  1. A mechanism for recording the outputs of the LLM in a fixture for mocking.
  2. A set of tests (interactive.test.ts) that start the interactive mode and ensure that it prompts the user. Tried to go further with /~https://github.com/microsoft/node-pty but ran into some snags.
  3. A full end-to-end test of the non-interactive CLI execution option, which internally calls a test command so that was tricky. This one also generates the implementation so it uses the LLM mock fixture.

Open to feedback/suggestions.

Copy link
Contributor

@steve8708 steve8708 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome @tpaulshippy! looks like just need to resolve a merge conflict

@steve8708 steve8708 merged commit a344018 into BuilderIO:main Jun 10, 2024
2 checks passed
@steve8708
Copy link
Contributor

🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants