-
-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add Pull Request model and integrate with sync_repository #845
Open
space-techy
wants to merge
26
commits into
OWASP:main
Choose a base branch
from
space-techy:feature/github-pull-request-tracking
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+393
−36
Open
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
0aff313
feat: Add Pull Request model and integrate with sync_repository
space-techy aa72a12
Merge branch 'main' into feature/github-pull-request-tracking
space-techy 1bffa45
Merge branch 'main' into feature/github-pull-request-tracking
space-techy 2a74139
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 022555a
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 180616d
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy ad3fe10
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy a1b81d9
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 8d1e84c
Add Pull Request Model and Sync repo
space-techy b601a62
Add Pull Request Mode and Improve Repository Model and Sync Repo
space-techy 093b6bc
Merge branch 'main' into feature/github-pull-request-tracking
space-techy 4b1d97f
resolve conflict with issue
space-techy 3989495
resolve conflict with issue
space-techy c787aa1
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 4d03eb0
Update code
arkid15r df52c43
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy bf4eaf7
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 0bbd86e
remove issue part
space-techy ef422c7
Fix conflict
space-techy 5268b4c
Fix conflict
space-techy 03a0fa3
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 0963487
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy c609b75
Merge branch 'OWASP:main' into feature/github-pull-request-tracking
space-techy 946bf40
Add Generic Issue and PR model
space-techy b43cbfe
Removed local migration
space-techy 4bd0594
Add Generic Model and Migrations
space-techy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
123 changes: 123 additions & 0 deletions
123
backend/apps/github/migrations/0017_alter_issue_managers_alter_issue_sequence_id_and_more.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
# Generated by Django 5.1.6 on 2025-02-28 20:27 | ||
|
||
import django.db.models.deletion | ||
import django.db.models.manager | ||
from django.db import migrations, models | ||
|
||
import apps.github.models.mixins.issue | ||
|
||
|
||
class Migration(migrations.Migration): | ||
dependencies = [ | ||
("github", "0016_user_is_bot"), | ||
] | ||
|
||
operations = [ | ||
migrations.AlterModelManagers( | ||
name="issue", | ||
managers=[ | ||
("open_issues", django.db.models.manager.Manager()), | ||
], | ||
), | ||
migrations.AlterField( | ||
model_name="issue", | ||
name="sequence_id", | ||
field=models.PositiveBigIntegerField(default=0, verbose_name="ID"), | ||
), | ||
migrations.AlterField( | ||
model_name="issue", | ||
name="state", | ||
field=models.CharField( | ||
choices=[("open", "Open"), ("closed", "Closed")], | ||
default="open", | ||
max_length=6, | ||
verbose_name="State", | ||
), | ||
), | ||
migrations.CreateModel( | ||
name="PullRequest", | ||
fields=[ | ||
( | ||
"id", | ||
models.BigAutoField( | ||
auto_created=True, primary_key=True, serialize=False, verbose_name="ID" | ||
), | ||
), | ||
("nest_created_at", models.DateTimeField(auto_now_add=True)), | ||
("nest_updated_at", models.DateTimeField(auto_now=True)), | ||
("node_id", models.CharField(unique=True, verbose_name="Node ID")), | ||
("title", models.CharField(max_length=1000, verbose_name="Title")), | ||
("body", models.TextField(default="", verbose_name="Body")), | ||
( | ||
"state", | ||
models.CharField( | ||
choices=[("open", "Open"), ("closed", "Closed")], | ||
default="open", | ||
max_length=6, | ||
verbose_name="State", | ||
), | ||
), | ||
("url", models.URLField(default="", max_length=500, verbose_name="URL")), | ||
("number", models.PositiveBigIntegerField(default=0, verbose_name="Number")), | ||
("sequence_id", models.PositiveBigIntegerField(default=0, verbose_name="ID")), | ||
( | ||
"closed_at", | ||
models.DateTimeField(blank=True, null=True, verbose_name="Closed at"), | ||
), | ||
("created_at", models.DateTimeField(verbose_name="Created at")), | ||
("updated_at", models.DateTimeField(db_index=True, verbose_name="Updated at")), | ||
( | ||
"merged_at", | ||
models.DateTimeField(blank=True, null=True, verbose_name="Merged at"), | ||
), | ||
( | ||
"assignees", | ||
models.ManyToManyField( | ||
blank=True, | ||
related_name="assigned_pull_requests", | ||
to="github.user", | ||
verbose_name="Assignees", | ||
), | ||
), | ||
( | ||
"author", | ||
models.ForeignKey( | ||
blank=True, | ||
null=True, | ||
on_delete=django.db.models.deletion.SET_NULL, | ||
related_name="created_pull_requests", | ||
to="github.user", | ||
verbose_name="Author", | ||
), | ||
), | ||
( | ||
"labels", | ||
models.ManyToManyField( | ||
blank=True, | ||
related_name="pull_request_labels", | ||
to="github.label", | ||
verbose_name="Labels", | ||
), | ||
), | ||
( | ||
"repository", | ||
models.ForeignKey( | ||
blank=True, | ||
null=True, | ||
on_delete=django.db.models.deletion.CASCADE, | ||
related_name="pull_requests", | ||
to="github.repository", | ||
), | ||
), | ||
], | ||
options={ | ||
"verbose_name_plural": "Pull Requests", | ||
"db_table": "github_pull_requests", | ||
"ordering": ("-updated_at", "-state"), | ||
}, | ||
bases=(apps.github.models.mixins.issue.IssueIndexMixin, models.Model), | ||
managers=[ | ||
("open_pull_requests", django.db.models.manager.Manager()), | ||
], | ||
), | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,3 @@ | ||
"""Github app.""" | ||
|
||
from .pull_request import PullRequest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
"""Github app Generic model.""" | ||
|
||
from django.db import models | ||
|
||
from apps.common.models import BulkSaveModel, TimestampedModel | ||
from apps.github.models.common import NodeModel | ||
from apps.github.models.mixins import IssueIndexMixin | ||
|
||
|
||
class GenericIssueModel(BulkSaveModel, IssueIndexMixin, NodeModel, TimestampedModel): | ||
"""Generic Issue And PR model.""" | ||
|
||
objects = models.Manager() | ||
|
||
class Meta: | ||
abstract = True | ||
|
||
class State(models.TextChoices): | ||
OPEN = "open", "Open" | ||
CLOSED = "closed", "Closed" | ||
|
||
title = models.CharField(verbose_name="Title", max_length=1000) | ||
body = models.TextField(verbose_name="Body", default="") | ||
|
||
state = models.CharField(verbose_name="State", max_length=6, choices=State, default=State.OPEN) | ||
url = models.URLField(verbose_name="URL", max_length=500, default="") | ||
|
||
number = models.PositiveBigIntegerField(verbose_name="Number", default=0) | ||
sequence_id = models.PositiveBigIntegerField(verbose_name="ID", default=0) | ||
|
||
closed_at = models.DateTimeField(verbose_name="Closed at", blank=True, null=True) | ||
created_at = models.DateTimeField(verbose_name="Created at") | ||
updated_at = models.DateTimeField(verbose_name="Updated at", db_index=True) | ||
|
||
def __str__(self): | ||
"""Issue human readable representation.""" | ||
return f"{self.title} by {self.author}" | ||
|
||
@property | ||
def project(self): | ||
"""Return project.""" | ||
return self.repository.project | ||
|
||
@property | ||
def repository_id(self): | ||
"""Return repository ID.""" | ||
return self.repository.id |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest applying the same logic we use for issues sync (see above). It fetches
all
states and has a cut off date if it's not the very first sync.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are talking about that
since
argument passed to **kwargs in issue?If you are then
since
is not available in PR!There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's try fetching PRs sorted by
updated
and break the loop when we see a date earlier than ourlatest_updated_at
(if any)?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can sort by the
created-at
field to retrieve the earliest date after the first fetch is complete. For subsequent runs, we can use this date. If the closed pull request’s date is older than the stored date, we can simply skip it.