Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CAI-63] Chatbot: create vector index from pipeline #1159

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

[CAI-63] Chatbot: create vector index from pipeline #1159

wants to merge 27 commits into from

Conversation

batdevis
Copy link
Collaborator

List of Changes

Addess "chatbot" action to deploy workflows.

Motivation and Context

We need to recreate LLamaindex Vector Index on Redis every time the documentation is recreated.

How Has This Been Tested?

I can't test it in local development.

@batdevis batdevis requested a review from a team as a code owner September 24, 2024 14:42
@changeset-bot changeset-bot
Copy link

changeset-bot bot commented Sep 24, 2024 •
edited
Loading

🦋 Changeset detected

Latest commit: 8281c3b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
infrastructure Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions GitHub Actions
Copy link
Contributor

This pull request is stale because it has been open for 14 days with no activity. If the pull request is still valid, please update it within 21 days to keep it open or merge it, otherwise it will be closed automatically.

@github-actions github-actions bot added the stale label Oct 15, 2024
@github-actions github-actions bot removed the stale label Oct 29, 2024
christian-calabrese
christian-calabrese reviewed Nov 4, 2024
View reviewed changes
.github/actions/chatbot/action.yaml Outdated Show resolved Hide resolved
.github/actions/chatbot/action.yaml Outdated Show resolved Hide resolved
.github/actions/chatbot/action.yaml Outdated Show resolved Hide resolved
.github/actions/chatbot/action.yaml Outdated Show resolved Hide resolved
.github/actions/chatbot/action.yaml Outdated Show resolved Hide resolved
.github/workflows/deploy_website.yaml Outdated Show resolved Hide resolved
.github/workflows/deploy_website_content.yaml Outdated Show resolved Hide resolved
.github/workflows/deploy_website_content.yaml Show resolved Hide resolved
.github/workflows/deploy_website.yaml Outdated Show resolved Hide resolved
.github/workflows/deploy_website.yaml Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

This pull request is stale because it has been open for 14 days with no activity. If the pull request is still valid, please update it within 21 days to keep it open or merge it, otherwise it will be closed automatically.

@github-actions github-actions bot added the stale label Nov 22, 2024
@github-actions github-actions bot removed the stale label Nov 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

This pull request is stale because it has been open for 14 days with no activity. If the pull request is still valid, please update it within 21 days to keep it open or merge it, otherwise it will be closed automatically.

@github-actions github-actions bot added stale and removed stale labels Dec 15, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 8, 2025

This pull request is stale because it has been open for 14 days with no activity. If the pull request is still valid, please update it within 21 days to keep it open or merge it, otherwise it will be closed automatically.

@github-actions github-actions bot added the stale label Jan 8, 2025
@github-actions github-actions bot added the infra label Jan 8, 2025
batdevis and others added 26 commits January 15, 2025 18:00
@batdevis @christian-calabrese
remove role
1c75e36 
his whole part of the pipeline will be run on a self hosted runner on AWS CodeBuild.
CodeBuild has its own role that can be used to contact AWS services
For this reason, it is not necessary to set IAM credentials/roles

Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
use commit id
c2c35d6 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
use commit id
edc2f30 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
use commit id
917a35b 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
remove AWS credentials
bb9e09a 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
runs on codebuild
80e7f99 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
remove AWS credentials
f912520 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
runs on AWS codebuild
95316a7 
This makes the entire step to run on the self-hosted runner in AWS. Also, this takes into account the target environment.

Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
remove AWS credentials
177187d 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
runs on AWS codebuild
9acf578 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
remove AWS credentials
622ce83 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
fix: pipeline
b00814d
@christian-calabrese
fix: runs-on set on the entire job
05e22e8
@christian-calabrese
feat: return lambda env variable
adc5b50
@christian-calabrese
fix: add shell bash
bd1c98a
@christian-calabrese
feat: add input chatbot_env_vars
0f24194
@batdevis @christian-calabrese
chore(chatbot): llama_index indipendent pipeline
716ddc3
@batdevis @christian-calabrese
ci(chatbot) create vector index pipeline
465b4ee
@batdevis @christian-calabrese
runs on codebuild
19f75e8 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
remove AWS credentials
abe51a1 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
runs on AWS codebuild
f3b52c0 
This makes the entire step to run on the self-hosted runner in AWS. Also, this takes into account the target environment.

Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@batdevis @christian-calabrese
remove AWS credentials
bb274be 
Co-authored-by: christian-calabrese <christian.calabrese@pagopa.it>
@christian-calabrese
fix: runs-on set on the entire job
6b46272
@christian-calabrese
feat: added chatbot env variables to codebuild project
8bcc92a
@batdevis @christian-calabrese
fix(chatbot): use python-setup image for amazon linux
d046803
@christian-calabrese
chore: ran terraform fmt and changeset added
8281c3b
@christian-calabrese christian-calabrese force-pushed the chatbot/llamaindex-pipeline branch from 52742a6 to 8281c3b Compare January 15, 2025 17:04
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 15, 2025 •
edited by jira bot
Loading

Jira Pull Request Link

This Pull Request refers to the following Jira issue CAI-63

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@christian-calabrese christian-calabrese christian-calabrese left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@batdevis batdevis

Labels
changeset infra size/small workflows
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@batdevis @christian-calabrese