Repository navigation
Dhruv - Fix blue square related issues - #2346
DeMoliT1on wants to merge 9 commits into
Conversation
- Stop Blue Squares for late joiners - Close enough hours email should also remove the blue square - Add full name to Administrative Details
one-community
added
the
High Priority - Please Review First
shubhamjakhete
left a comment
shubhamjakhete
left a comment
There was a problem hiding this comment.
Reviewed the changes. The Wednesday-or-later new-user handling is updated correctly, the close-enough-hours flow removes the associated Blue Square, and the user's full name is now included in the Administrative Details section. I did not find any blocking issues with the implementation. Looks good to me.
Isha2303165
left a comment
Isha2303165
left a comment
There was a problem hiding this comment.
Tested the changed backend helpers and all 110 focused tests passed. However, I found issues in the new Blue Square/email flow that should be addressed before approval. During assignBlueSquareForTimeNotMet, the resolved email recipients are showing as Promises, for example: “Email BCCs for blue square assignment: Promise { }”, rather than resolved arrays. This indicates the async CC/BCC helpers may be used without awaiting them.
I also noticed that the new close-enough-hours flow calls getUserRoleByEmail(user), which iterates over user.teams, and also uses user.lastName, while the user projection in weeklyAutoReplyEmailFunction does not appear to include teams or lastName. This could cause the new path to fail with real queried user documents.
The test run also logs a caught error, “TypeError: original.toObject is not a function”, from weeklyAutoReplyEmailFunction even though the suite reports passing. Please address these runtime issues and add or adjust tests so the full success path completes without errors.
Isha2303165
left a comment
Isha2303165
left a comment
There was a problem hiding this comment.
Rechecked the latest updates after my previous review. All 110 focused tests across 3 suites passed, and the backend build completed successfully with 767 files compiled. The previously observed issues also appear resolved: Blue Square email BCC values are now resolved correctly instead of remaining pending Promises, the weekly auto-reply query includes the required lastName and teams fields, and the previous toObject runtime error did not recur during testing. No blocking issues found in the updated changes. Approved.
AaditTrivedi
left a comment
AaditTrivedi
left a comment
There was a problem hiding this comment.
Reviewed by Aadit Trivedi.
How I verified
Checked out the PR locally (Node 22), installed with npm ci from package-lock.json, ran the three affected suites, and read the changes in userHelper.js against the description and @Isha2303165's review. I did not run the cron flow end to end against a database.
Note for other reviewers: installing with yarn makes userHelper.spec.js and getInfringementEmailBody.test.js fail to load (htmlparser2 is ESM-only). The same failure happens on development, so it is pre-existing and not caused by this PR. yarn.lock is out of sync with package.json on development too. Use npm ci to run these tests.
Verified working
- Tests:
userHelper.spec.js,warningsHelper.spec.js, andgetInfringementEmailBody.test.jspass 110/110. Screenshot below. - @Isha2303165's three issues are resolved:
- No
Promise { <pending> }recipients appear in the test output. - The
weeklyAutoReplyEmailFunctionuser query now includeslastNameandteams(line 1299), which the close-enough path uses. - No
toObject is not a functionerror appears in the test output.
- No
- Wednesday rule: both checks moved from
> 1to> 2, so only start dates of Wednesday or later get a pass, matching the description. - The user's full name appears in the Administrative Details section of the email.
Issues found
- Scope is much larger than described. The description lists
userHelper.jsand the test script, but the PR also moves about 250 lines fromwarningsController.jsinto a newwarningsHelper.js, bumpsmongodb-memory-serverfrom ^7.6.3 to ^11.2.0 (four major versions, affecting every test that uses it), rewrites large parts of both lockfiles, and editssonar-project.properties. Please document these or move them to separate PRs. sonar-project.propertiesaddssrc/scripts/**tosonar.exclusions. That excludes every script in the repo from the quality gate, including the test script this PR edits, so its issues are hidden rather than fixed. Please revert this and address the issues directly.- Jae's name and email are hardcoded in
monitorData(jae@onecommunityglobal.org). Please read the monitor contact from configuration or an existing lookup rather than code. - Conflict with #2369: this PR moves
filterWarningsout ofwarningsController.js, and #2369 modifiesfilterWarningsin that same file to return tracker descriptions. Whichever merges second needs to carry the other's changes over, or the description field will be lost. Worth coordinating with @AnthonyWeathers.
Pointers, not blocking
- The warning tracker query (
currentWarnings.find({ activeWarning: true }, ...)) runs inside the per-user loop, so it repeats for every qualifying user. It returns the same data each time and can be fetched once before the loop. if (typeof currentWarningDescriptions !== 'undefined')is always true right after theawait, so the check and its comment can be removed.- Pre-existing, not from this PR: around line 380 the sort compares
a.lastNamewithb.lastname(lowercase n), so the second name always ends in "undefined."
Evidence
Test run, 110/110 passing with npm ci:
Required action
Please revert the Sonar exclusion, document or split out the extra changes, move the monitor contact out of the code, and coordinate with #2369. Happy to re-review once updated.
manavkheni1
left a comment
manavkheni1
left a comment
There was a problem hiding this comment.
Reviewed by Manav. Pulled the branch locally (npm ci, per @AaditTrivedi's note about npm install failing on htmlparser2), and independently verified the following:
Tests: Ran the same three suites (userHelper.spec.js, warningsHelper.spec.js, getInfringementEmailBody.test.js) — 110/110 passing, no Promise { } values, no toObject is not a function errors.
I also confirmed @AaditTrivedi's four outstanding concerns are still present and should be addressed before merge:
- mongodb-memory-server bumped from ^7.6.3 to ^11.2.0 (4 major versions) — unrelated to this PR's stated scope
- sonar-project.properties now excludes src/scripts/** from the quality gate, hiding rather than fixing issues in the test script
- Hardcoded email "jae@onecommunityglobal.org" at userHelper.js line 173, inside getUserRoleByEmail — should come from config or an existing lookup instead
- Merge conflict risk with #2369, which also modifies filterWarnings/warningsController.js
Agreeing changes should be addressed before this merges. Screenshot below showing my test run.
…config * Update sonar-project.properties to use sonar.coverage.exclusions for src/scripts instead of full exclusion * Fix a pre-existing case-sensitivity sorting bug using lastName instead of lastname in userHelper.js * Optimize performance by moving the currentWarningDescriptions query outside of the user loop in userHelper.js * Simplify warningId lookup using optional chaining and remove redundant undefined checks * Replace hardcoded monitor contact details in monitorData with MONITOR_CONTACT_CONFIG
Thanks for the detailed review! Here is what I've updated:
|
vidiyala99
left a comment
vidiyala99
left a comment
There was a problem hiding this comment.
Reviewed d2b0143 (no review on this head yet). I ran the PR's suites offline (no DB, no .env): userHelper.spec.js, warningsHelper.spec.js and getInfringementEmailBody.test.js pass 110/110. I also started this branch's backend and checked Dashboard > Show Trackers as Admin: 67 GET /api/warnings/:id calls succeeded and the trackers render as before (including the "Blu Sq Rmvd - Hrs Close Enoug" row), so the warningsHelper extraction looks behavior-preserving. The Wednesday start-day rule and the admin Name: line match the description, and the earlier review points (hoisted warning query, lastName typo) are addressed.
I did not run testEmailJobs.js against the shared dev DB, because of point 1. Instead I wrote three small mocked unit tests (below, collapsed); each one passing demonstrates one of the first three issues:
- Targeted test runs are not isolated. With
targetUserIdset,assignBlueSquareForTimeNotMetscopes only the active-user query (userHelper.js:886-888). The inactive-user block (:954-962) still runsprocessWeeklySummariesByUserIdfor every inactive user, which pushes an empty summary with$slice: 4and drops their oldest stored summary, anddeleteOldTimeOffRequests()(:938) still deletes expired time-off requests for everyone. Following test step 7 on the shared dev DB therefore changes data for users who are not the target (proof P1). Please skip both blocks when a target is given, and consider a dry-run flag for the script. - Auto-removal also removes blue squares earned for a missing summary. A user at 85 to 99 percent of their hours with no summary still resolves to
MISSED_HOURS_BY_<15%(resolveAutoReplyTemplatehas no summary input; onlymetHours && !hasSummaryis skipped at:1325), and the new code then pulls their blue square (:1347-1349), whose description cites both reasons (proof P2). Please only remove it when the summary was submitted. - The
$pullmatches on date only.$pull: { infringements: { date: assignmentDate } }removes every blue square dated that Sunday, including manually assigned ones, although the comment says "system-assigned" (proof P3). AddingmanuallyAssigned: { $ne: true }(the field exists on the schema) would match the intent. The current test steps use a manual blue square, so they pass for the wrong reason. - Not idempotent. Each run pushes another warning, and on the fourth-plus path a second run would pull the re-issued blue square (same date) and issue another.
infringementCountis also left stale after the pull in the blue and yellow cases. - Contradictory emails in production. A fourth-offense user gets "issue blue square" (
:1412), then "New Infringement Assigned" (:1446), then the "close enough for us to remove this blue square" template (:1460). These new sends also ignore the email/cc/bcc overrides, andsendEmailToUseris not awaited. - The monitor contact is still hardcoded:
monitorContactConfig.jsholds literals, andgetUserRoleByEmail(:174) andsendBlueSquareEmail(:1174-1176) still embed the address. Themongodb-memory-serverbump and lockfile churn are also still in, and the conflict with #2369 remains.
Proof tests (mocked models, no DB). Save as src/helpers/__tests__/PREP-2346.proof.spec.js and run npx jest src/helpers/__tests__/PREP-2346.proof.spec.js
const mongoose = require('mongoose');
const moment = require('moment-timezone');
/* =======================
MOCKS (MUST COME FIRST)
======================= */
jest.mock('../../models/userProfile', () => ({
findById: jest.fn(),
find: jest.fn(),
aggregate: jest.fn(),
updateOne: jest.fn().mockResolvedValue({}),
findByIdAndUpdate: jest.fn((id, update, cb) => {
if (typeof cb === 'function') cb(null);
return Promise.resolve({});
}),
}));
jest.mock('../../models/badge', () => ({
find: jest.fn(),
findOne: jest.fn(),
}));
jest.mock('../../models/team', () => ({
aggregate: jest.fn(),
}));
jest.mock('../../utilities/timeUtils');
jest.mock('../../utilities/emailSender');
jest.mock('../dashboardhelper', () =>
jest.fn(() => ({
laborthisweek: jest.fn().mockResolvedValue([{ timeSpent_hrs: 36 }]),
})),
);
jest.mock('../../models/BlueSquareEmailAssignment', () => ({
find: jest.fn().mockImplementation(() => ({
populate: jest.fn().mockImplementation(() => ({
exec: jest.fn().mockResolvedValue([
{
email: 'bcc-test@example.com',
assignedTo: { isActive: true },
},
]),
})),
})),
}));
jest.mock('../../models/timeOffRequest', () => ({
deleteMany: jest.fn(),
find: jest.fn(),
}));
/* =======================
IMPORTS AFTER MOCKS
======================= */
const userProfile = require('../../models/userProfile');
const currentWarnings = require('../../models/currentWarnings');
const warningsHelper = require('../warningsHelper');
const userHelperFactory = require('../userHelper');
const emailSender = require('../../utilities/emailSender');
const { COMPANY_TZ } = require('../../constants/company');
const timeOffRequest = require('../../models/timeOffRequest');
const { weeklyAutoReplyEmailFunction, assignBlueSquareForTimeNotMet } = userHelperFactory();
/* ===== PREP-2346 proof tests (reviewer-only, not part of PR) ===== */
describe('PREP-2346 proofs', () => {
const WARNING_DESC = 'Blu Sq Rmvd - Hrs Close Enoug';
const assignmentDate = moment().tz(COMPANY_TZ).startOf('week').format('YYYY-MM-DD');
beforeEach(() => {
jest.clearAllMocks();
emailSender.mockResolvedValue(true);
});
it('P1: targeted assignBlueSquareForTimeNotMet still sweeps ALL inactive users weeklySummaries', async () => {
userProfile.find
.mockResolvedValueOnce([]) // target user query
.mockResolvedValueOnce([{ _id: new mongoose.Types.ObjectId() }, { _id: new mongoose.Types.ObjectId() }]); // inactive
userProfile.findByIdAndUpdate.mockClear();
await assignBlueSquareForTimeNotMet({
targetUserId: new mongoose.Types.ObjectId(),
ccOverride: ['cc@x.com'],
bccOverride: ['bcc@x.com'],
});
expect(userProfile.find).toHaveBeenNthCalledWith(2, { isActive: false }, '_id');
const summaryShifts = userProfile.findByIdAndUpdate.mock.calls.filter(
(c) => c[1] && c[1].$push && c[1].$push.weeklySummaries,
);
expect(summaryShifts.length).toBe(2); // both inactive users had summaries shifted ($slice: 4 drops oldest)
expect(summaryShifts[0][1].$push.weeklySummaries.$slice).toBe(4);
expect(timeOffRequest.deleteMany).toHaveBeenCalled(); // global time-off cleanup also runs
});
it('P2: 85-99% hours user WITHOUT a weekly summary still gets the blue square pulled', async () => {
jest.spyOn(timeOffRequest, 'find').mockResolvedValue([]);
jest.spyOn(currentWarnings, 'find').mockReturnValue({
sort: jest.fn().mockResolvedValue([{ warningTitle: WARNING_DESC, _id: 'w1' }]),
});
jest.spyOn(warningsHelper, 'filterWarnings').mockReturnValue({ sendEmail: null, size: 0 });
jest.spyOn(warningsHelper, 'sendEmailToUser').mockImplementation(() => {});
const user = {
_id: '60c72b2f9b1d8b2bad709999',
email: 'nosummary@example.com',
firstName: 'No',
lastName: 'Summary',
weeklycommittedHours: 40, // dashboardhelper mock returns 36h = 90% of 40
missedHours: 0,
startDate: '2022-01-01',
weeklySummaryOption: 'Required',
weeklySummaryNotReq: false,
weeklySummaries: [{ summary: '' }, { summary: '' }], // NO summary last week
infringements: [
{ date: assignmentDate, description: 'System auto-assigned infringement for two reasons: not meeting weekly volunteer time commitment as well as not submitting a weekly summary.' },
],
teams: [],
warnings: [],
};
userProfile.find.mockResolvedValueOnce([user]);
userProfile.findByIdAndUpdate.mockReset();
userProfile.findByIdAndUpdate.mockResolvedValue(user);
await weeklyAutoReplyEmailFunction({ targetUserId: user._id, bccOverride: ['b@x.com'] });
expect(userProfile.findByIdAndUpdate).toHaveBeenCalledWith(user._id, {
$pull: { infringements: { date: assignmentDate } },
});
const body = emailSender.mock.calls.map((c) => c[2]).join(' ');
expect(body).toContain('close enough to your total hours for us to remove this blue square');
});
it('P3: $pull is by date only, so a MANUAL blue square on the same Sunday is removed too', async () => {
jest.spyOn(timeOffRequest, 'find').mockResolvedValue([]);
jest.spyOn(currentWarnings, 'find').mockReturnValue({ sort: jest.fn().mockResolvedValue([]) });
jest.spyOn(warningsHelper, 'filterWarnings').mockReturnValue({ sendEmail: null, size: 0 });
const user = {
_id: '60c72b2f9b1d8b2bad708888', email: 'm@example.com', firstName: 'M', lastName: 'A',
weeklycommittedHours: 40, missedHours: 0, startDate: '2022-01-01',
weeklySummaryOption: 'Required', weeklySummaries: [{}, { summary: 'ok' }],
infringements: [
{ date: assignmentDate, description: 'system hours' },
{ date: assignmentDate, description: 'manual: missed video call', manuallyAssigned: true },
],
teams: [], warnings: [],
};
userProfile.find.mockResolvedValueOnce([user]);
userProfile.findByIdAndUpdate.mockReset();
userProfile.findByIdAndUpdate.mockResolvedValue(user);
await weeklyAutoReplyEmailFunction({ targetUserId: user._id, bccOverride: ['b@x.com'] });
const pull = userProfile.findByIdAndUpdate.mock.calls.find((c) => c[1].$pull);
expect(pull[1].$pull.infringements).toEqual({ date: assignmentDate }); // no manuallyAssigned / description filter
});
});02: the three mocked proof tests pass on d2b0143; each pass demonstrates issue 1, 2 or 3
03a: targetUserId only scopes the active-user query; time-off cleanup and the inactive sweep stay global
03b: the near-miss path skips only met-hours users without a summary, and the pull is keyed on date alone
01: the PR's own suites pass 110/110 (working as intended)
04 (Admin, light): Show Trackers works on this branch's backend (working as intended)
adit24dhaya
left a comment
adit24dhaya
left a comment
There was a problem hiding this comment.
Re-reviewed the current head d2b0143 after the remediation commit.
I verified the latest changes locally without a database: the three affected suites passed (3/3 suites, 110/110 tests), and npm run build compiled 775 files successfully. I also confirmed that the Sonar exclusion is now limited to coverage, the warning query was hoisted, and the lastName sort typo was corrected.
Changes are still required before merge. The monitor contact was moved to monitorContactConfig.js, but the name and email remain hardcoded there, so the original configurability concern is not resolved. More importantly, the current head still has the same production-behavior problems documented with mocked proofs in the existing same-head review: targeted runs can mutate unrelated users/global data, the close-enough path can remove a blue square when the summary is missing, and the date-only $pull can remove manual infringements. I confirmed those paths remain present in userHelper.js.
I’m not adding duplicate inline comments because the existing review on this exact commit already provides reproducible proof tests and precise locations. Please address those current-head findings and add regression coverage for the targeted/dry-run boundary and selective infringement removal.
|
@vidiyala99 @adit24dhaya Have included the required changes for the PR. Also would like to add that the monitor contact config is temporarily hardcoded, and the responsible DevOps team can convert it to env variables since I do not have permission to add the same for Dev/Production, hence have left a note in the config file. I have also coordinated with @AnthonyWeathers in regards to #2369 that once one of the PR gets merged, the other would incorporate the required changes while handle the merge conflicts. |
56952f2 to
bd27455
Compare
|
RichaSapre
left a comment
RichaSapre
left a comment
There was a problem hiding this comment.
Tested PR #2346 locally using npm ci. The three affected test suites passed with 110/110 tests, and npm run build completed successfully with 775 files compiled.
I reviewed the updated Blue Square assignment and weekly auto-reply flows, including the new handling for close-enough hours, warning creation, infringement updates, and targeted-user testing.
I found one remaining correctness issue: manually assigned Blue Squares are still considered eligible by hasTodayBlueSquare. Although the later $pull excludes manually assigned infringements, the function can still create a “Removed Blue Square” warning and send the removal email while the manually assigned square remains. Please restrict this flow to system-assigned squares and add a regression test.
| const userAfterPull = await userProfile.findByIdAndUpdate( | ||
| user._id, | ||
| { | ||
| $pull: { infringements: { date: assignmentDate, manuallyAssigned: { $ne: true } } }, |
There was a problem hiding this comment.
This now filters manually assigned infringements out of the $pull, but hasTodayBlueSquare still treats a manually assigned square as eligible for the close-enough flow. The function can therefore add the “Removed Blue Square” warning and send the removal email even though the manually assigned square remains. Could we require a system-assigned square here as well and add a regression test for this case?
AaditTrivedi
left a comment
AaditTrivedi
left a comment
There was a problem hiding this comment.
Re-reviewed by Aadit Trivedi.
Follow-up to my earlier review, checking the commits from Sep 28 and Oct 3 ("sonar exclusions, optimize warning queries, and extract monitor config" and "Address reviewer comments") against the diff with development.
Fixed since my review
- Sonar:
src/scripts/**is no longer insonar.exclusions; it is now only insonar.coverage.exclusions, so scripts are still analyzed for issues. That addresses my concern. - The monitor contact is now read from the new
src/config/monitorContactConfig.jsinstead of being hardcoded in the warning logic.
Still open
mongodb-memory-serveris still bumped from ^7.6.3 to ^11.2.0. If the upgrade is needed for these tests, please explain why in the description; otherwise please move it to a separate PR, since it affects every test that uses an in-memory database.- A new hardcoded recipient was added:
const recipients = ['jae@onecommunityglobal.org'];(around line 174 ofuserHelper.js). Since the monitor contact now lives inmonitorContactConfig.js, please read this from the same config. The other occurrences of that address in the file were already on development, so they are outside this PR.
Pointer, not blocking
- Please confirm coordination with #2369, since both PRs still change
filterWarnings.
Required action
Please address the two open points. The rest of my earlier review is resolved.
This branch has not been deployed
9 participants
Description
This PR fixes few of the Blue Square assignment issues:
Related PRS (if any):
This is a backend only PR
Main changes explained:
warningsController.jsinto a newwarningsHelper.jsfile. These helper functions support the Warning Logging system i.e automating the process where admins previously had to manually remove close-enough blue squares from the UI, while still properly logging the warning and notifying both the user and the admin via email.userHelper.jsto handle new user logic, append full names to Administrative Details, and automate the removal of blue squares for close-enough hours within the updated cron job function.sonar-project.propertiesto usesonar.coverage.exclusionsinstead of a full script exclusion, allowing quality gate security analyses while ignoring irrelevant test script coverage.testEmailJobs.jsto manually test cron job functions modified in this PR.How to test:
MISSED_BY_<15%i.e no of hours logged in previous week is 85-99% of the weekly commitment./userprofile/<target_user_id>testEmailJobs.jsscript to call theassignBlueSquareForTimeNotMetfunction. Please make sure to changeTARGET_USER_IDto the id from userporfile page ``/userprofile/<target_user_id> before calling the function.weeklyAutoReplyEmailFunctionfrom the test script with the same user id.Screenshots or videos of changes:
Blue.Square.Fix.Summary.Video.webm
Note:
Since the changes in the PR are related to cron job functions which runs weekly, the test script is the only way to test the changes. Additionally you won't be able to receive any infringement emails since those are only possible on Production.