Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DCMAW-10799: Enable Dequeue Lambda to write TxMA events into new Dequeue table #321

Merged
merged 112 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
361b87c
provision dequeue table and permissions for dequeue lambda
jhumbert-dd Dec 10, 2024
b3f54d1
add ttl to sam template
jhumbert-dd Dec 11, 2024
d185436
handle parsing errors when processing evnents
jhumbert-dd Dec 12, 2024
63e2115
WIP - handle message processing failures
jhumbert-dd Dec 12, 2024
237e158
handle messages that fail to be processed
jhumbert-dd Dec 13, 2024
fa8db27
add and update tests
jhumbert-dd Dec 13, 2024
1b57711
remove batch item failures and add console log
jhumbert-dd Dec 13, 2024
a6ef239
update test descriptions
jhumbert-dd Dec 13, 2024
4099831
update test assertions
jhumbert-dd Dec 13, 2024
af916e6
update deploy script to exit if build fails
jhumbert-dd Jan 6, 2025
e09902e
add txma and credential types
jhumbert-dd Jan 6, 2025
c195afc
add dynamodb dependencies
jhumbert-dd Jan 6, 2025
642952d
write txma events to test-resources dynamodb table
jhumbert-dd Jan 6, 2025
17a3c44
extract dynamodb client into an adapter and update tests
jhumbert-dd Jan 6, 2025
b266414
update put request
jhumbert-dd Jan 7, 2025
4a00f5f
update error handling when unable to parse record body
jhumbert-dd Jan 7, 2025
f07d544
update test with processing failures
jhumbert-dd Jan 7, 2025
2f880f7
move adapters folder
jhumbert-dd Jan 7, 2025
f8b9d87
add logger
jhumbert-dd Jan 7, 2025
890df64
add logger and db adapter to dequeue handler dependencies
jhumbert-dd Jan 7, 2025
15a003d
tidy files
jhumbert-dd Jan 7, 2025
66877a6
return batch item failures to be retried
jhumbert-dd Jan 7, 2025
0dfb64a
minor changes
jhumbert-dd Jan 7, 2025
bc41edf
Merge branch 'main' into DCMAW-10799
jhumbert-dd Jan 7, 2025
54dc2a0
amend import
jhumbert-dd Jan 7, 2025
bb50616
remove unused const
jhumbert-dd Jan 7, 2025
d158264
disable next line where type any is used
jhumbert-dd Jan 7, 2025
8027c45
apply sonar fixes
jhumbert-dd Jan 7, 2025
7113282
format and lint
jhumbert-dd Jan 7, 2025
614a7d7
move dynamo client back into Lambda and use mockClient
jhumbert-dd Jan 7, 2025
3c56149
add dequeue table name stack output
jhumbert-dd Jan 8, 2025
c02598b
add dequeue table name env var and build env script
jhumbert-dd Jan 8, 2025
7ef23ea
format
jhumbert-dd Jan 8, 2025
fc93ac2
remove unused log message names
jhumbert-dd Jan 8, 2025
066e317
update message code prefix
jhumbert-dd Jan 8, 2025
8189add
Merge branch 'main' into DCMAW-10799
jhumbert-dd Jan 8, 2025
36f31a5
add time to live
jhumbert-dd Jan 8, 2025
0b04e3b
rename Dynamo client
jhumbert-dd Jan 8, 2025
1e0a953
minor change
jhumbert-dd Jan 8, 2025
4ab8cdc
update logger and mockLogger
jhumbert-dd Jan 8, 2025
cccec4f
set service name
jhumbert-dd Jan 8, 2025
d4c125a
simplify logic
jhumbert-dd Jan 8, 2025
7b328bc
format
jhumbert-dd Jan 8, 2025
9723eba
update table name and env var in SAM template
jhumbert-dd Jan 8, 2025
180eb34
apply sonar fix
jhumbert-dd Jan 8, 2025
906819e
remove condition from template
jhumbert-dd Jan 8, 2025
e8b88d3
update log message type
jhumbert-dd Jan 9, 2025
1c178ce
update logger test file
jhumbert-dd Jan 9, 2025
0b3f156
remove common logs and move to registered logs file
jhumbert-dd Jan 9, 2025
1618ae6
use lambda context
jhumbert-dd Jan 9, 2025
d1199ef
update txma event type
jhumbert-dd Jan 9, 2025
9f2403f
rename dynamodb table
jhumbert-dd Jan 9, 2025
b26634a
rename message code
jhumbert-dd Jan 9, 2025
7fa9c31
rename default stack name in generate_env_file.sh
jhumbert-dd Jan 9, 2025
50f233b
amend message code in logger tests
jhumbert-dd Jan 9, 2025
d5f9bef
add event name validation
jhumbert-dd Jan 9, 2025
b94470b
format
jhumbert-dd Jan 9, 2025
574a481
update dynamo command to PutItemCommand
jhumbert-dd Jan 9, 2025
143e668
add message validation
jhumbert-dd Jan 9, 2025
fd0d5a0
extract message validation
jhumbert-dd Jan 9, 2025
20b6f0f
update generate env file
jhumbert-dd Jan 9, 2025
25c5ee8
remove generate env file
jhumbert-dd Jan 9, 2025
256673f
extract env var validation
jhumbert-dd Jan 9, 2025
ff8cf67
remove errorCategory from logs
jhumbert-dd Jan 10, 2025
5b7964f
update ttl and dynamo action in template
jhumbert-dd Jan 10, 2025
43e4ecb
update dynamo key format
jhumbert-dd Jan 10, 2025
bc990c3
update key name and add keys to dynamo error log
jhumbert-dd Jan 10, 2025
3af78d8
DCMAW-10799: move test file into tests folder
sandymay-dd Jan 10, 2025
415f87a
DCMAW-10799: Start extracting test data for readibility
sandymay-dd Jan 10, 2025
4791554
add processedMessages assertions
jhumbert-dd Jan 10, 2025
95be83b
DCMAW-10799: test data for missing event name
sandymay-dd Jan 10, 2025
fcc4b35
DCMAW:10799: Update event_name not allowed test data and assertion
sandymay-dd Jan 10, 2025
0909ce0
DCMAW:10799: Update missing user not allowed test data and assertion
sandymay-dd Jan 10, 2025
74cefda
DCMAW:10799: Update missing sessionId not allowed test data and asse…
sandymay-dd Jan 10, 2025
d08c7c6
DCMAW:10799: Update invalid sessionIdtest data and assertion
sandymay-dd Jan 10, 2025
851a4b7
DCMAW:10799: Update missing timestamp data and assertion
sandymay-dd Jan 10, 2025
5cb0564
add batchItemFailure assertions and use variables for test data
jhumbert-dd Jan 10, 2025
3f8ade2
DCMAW-10799: Test data for passed Dynamo queries and failed messageId
sandymay-dd Jan 10, 2025
1cc112e
Update from merge conflict and start adding more test data consts
sandymay-dd Jan 10, 2025
554bc8f
update test values with test data variables
jhumbert-dd Jan 10, 2025
f9e29a9
update processedMessages type
jhumbert-dd Jan 10, 2025
2f21708
Merge branch 'main' into DCMAW-10799
jhumbert-dd Jan 10, 2025
c8c53dd
minor changes
jhumbert-dd Jan 10, 2025
9b2fbd2
update test mocks
jhumbert-dd Jan 10, 2025
41fbc5e
add message code assertions
jhumbert-dd Jan 10, 2025
281121b
move test data into a beforeEach block
jhumbert-dd Jan 10, 2025
ad57f5f
update test case
jhumbert-dd Jan 10, 2025
a61cbd5
DCMAW-10799: Guidance for restructuring tests
sandymay-dd Jan 10, 2025
ebd109c
restructure tests
jhumbert-dd Jan 10, 2025
a631272
DCMAW-10799: Allow events without sessionId in schema to be processed
sandymay-dd Jan 10, 2025
344f231
update test data
jhumbert-dd Jan 10, 2025
d75477b
restruture tests
jhumbert-dd Jan 10, 2025
28f7f67
minor changes
jhumbert-dd Jan 12, 2025
116919f
wip - add getEvent function as a dependency
jhumbert-dd Jan 13, 2025
9bdf7c0
remove batchItemFailures after event validation
jhumbert-dd Jan 13, 2025
d98adfb
lint
jhumbert-dd Jan 13, 2025
e70ef71
remove duplicate line
jhumbert-dd Jan 13, 2025
a8322db
address sonar issue
jhumbert-dd Jan 13, 2025
212a8a1
Merge branch 'main' into DCMAW-10799
jhumbert-dd Jan 13, 2025
3a34beb
remove stack output
jhumbert-dd Jan 13, 2025
66ad235
remove unused type
jhumbert-dd Jan 13, 2025
ded89ee
remove export from individual test data record values
jhumbert-dd Jan 13, 2025
32b5987
Replace test data string with variable
jhumbert-dd Jan 13, 2025
727ccaf
update mockDbClient to use rejects as default
jhumbert-dd Jan 13, 2025
9ccc804
remove duplicate assertion
jhumbert-dd Jan 13, 2025
bb4f2c1
update test decriptions
jhumbert-dd Jan 13, 2025
4c28f9d
update logging readme
jhumbert-dd Jan 13, 2025
de7b889
remove comment
jhumbert-dd Jan 13, 2025
2c79a30
Merge branch 'DCMAW-10799-extract-getEvent' into DCMAW-10799
jhumbert-dd Jan 13, 2025
a2a00af
add assertions to happy path tests
jhumbert-dd Jan 13, 2025
67d625b
extract getEvent function
jhumbert-dd Jan 13, 2025
b137702
Merge branch 'main' into DCMAW-10799
jhumbert-dd Jan 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions helper-scripts/deploy_test_resources.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,17 @@ while true; do
cd ../test-resources
npm i
sam build --cached
sam deploy \
--stack-name "$TEST_RESOURCES_STACK_NAME" \
--parameter-overrides "BackendApi=${BACKEND_STACK_NAME}" \
--capabilities CAPABILITY_NAMED_IAM \
--resolve-s3
if [ $? -ge 1 ]; then
echo "Build failed"
exit 1
else
sam deploy \
--stack-name "$TEST_RESOURCES_STACK_NAME" \
--parameter-overrides "BackendApi=${BACKEND_STACK_NAME}" \
--capabilities CAPABILITY_NAMED_IAM \
--resolve-s3
fi

break
;;
[nN])
Expand Down
18 changes: 18 additions & 0 deletions test-resources/generate_env_file.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash
set -eu

STACK_NAME_SUFFIX="-test-resources"

if [ $# -ge 1 ] && [ -n "$1" ] ; then
STACK_NAME="${1}${STACK_NAME_SUFFIX}"
else
STACK_NAME="mob-async-backend${STACK_NAME_SUFFIX}" # default stack name in dev
jhumbert-dd marked this conversation as resolved.
Show resolved Hide resolved
fi

echo "Generating .env file for the $STACK_NAME stack"

EVENTS_TABLE_NAME=$(aws cloudformation describe-stacks --stack-name "$STACK_NAME" --query "Stacks[0].Outputs[?OutputKey=='EvnetsTableName'].OutputValue" --output text)
jhumbert-dd marked this conversation as resolved.
Show resolved Hide resolved

echo "EVENTS_TABLE_NAME=$EVENTS_TABLE_NAME" > .env
echo "TXMA_EVENT_TTL_DURATION_IN_SECONDS=3600" >> .env

Loading