Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

change daily job to process latest events #982

Merged
merged 2 commits into from
Nov 21, 2024
Merged

change daily job to process latest events #982

merged 2 commits into from
Nov 21, 2024

Conversation

adnahassan
Copy link
Contributor

@adnahassan adnahassan commented Oct 22, 2024 •
edited
Loading

These changes are in reference to this spike

Changes in this PR

DPT-622

  • add processed_time stage layer table attribute
  • change logic to retrieve latest data in to stage layer. Receives latest events now instead of events from day before
  • add a second step which processes any event_ids missed by previous run

DPT-623

  • add hypercare eventbridge rule to run at 8:30, 11:30, and 14:30

misc

  • Added new set up scripts for redshift

@adnahassan adnahassan requested review from a team as code owners October 22, 2024 12:16
nacnudus
nacnudus previously approved these changes Oct 22, 2024
View reviewed changes
@adnahassan adnahassan force-pushed the dpt-622 branch 2 times, most recently from b0ff00a to 471cb27 Compare November 5, 2024 13:54
@adnahassan adnahassan merged commit 2306723 into main Nov 21, 2024
2 checks passed
@adnahassan adnahassan deleted the dpt-622 branch November 21, 2024 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richa-misra-gds richa-misra-gds richa-misra-gds approved these changes

@nacnudus nacnudus nacnudus left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@adnahassan @nacnudus @richa-misra-gds