submit-social-housing-lettings-and-sales-data

Commit Graph

Author	SHA1	Message	Date
Samuel Young	d79cfcd415	CLDC-4237: Upgrade ruby version (#3190 ) * CLDC-4237: Upgrade ruby to 3.4.4 * CLDC-4237: Add additional build dependencies to Dockerfile * CLDC-4237: Update rubocop * CLDC-4237: Fix new rubocop rules most auto applies RSpec/IndexedLet had 200 hits needing manual correction so left these for now * CLDC-4237: Resolve vulnerable dependencies * CLDC-4237: Fix erb files * CLDC-4237: Downgrade connection_pool	3 weeks ago
Nat Dean-Lewis	02f319396b	CLDC-4140: remove old gender question lettings (#3177 ) * CLDC-4141: copy person questions refactor from CLDC-4142 * CLDC-4141: remove gender same as sex and replace gender identity q * CLDC-4141: replace sex with sexrab where relevant for 2026 * CLDC-4141: update download order * CLDC-4140: remove sex from rowparser entirely * CLDC-4140: keep sex in factory for now * CLDC-4140: update household characteristics test * CLDC-4140: update log variable spec * CLDC-4140: update csv export tests * CLDC-4140: update validator tests * CLDC-4140: update row-parser spec * CLDC-4140: update csv-parser spec * CLDC-4140: update export spec * CLDC-4140: update log var defs spec * CLDC-4140: update export spec * CLDC-4140: update row parser spec * CLDC-4140: update validator spec * CLDC-4140: update csv parser spec * CLDC-4143: Update gender retirement validations rename functions as the new wording is more related to those that identify as male and any other option * CLDC-4143: Add 2026 tests * CLDC-4143: Update legacy json files causes issues with some old tests * CLDC-4143: Add error mappings for gender fields * CLDC-4143: Add new gender age retirement soft validation replaces the many check pages from previous years with a single one shows if either of the previous ones would show has agreed wording on ticket * fixup! CLDC-4143: Update gender retirement validations improve gender comment wording * CLDC-4143: Update non males validation name reference to 'soft validation' we found confusing * fixup! CLDC-4143: Add new gender age retirement soft validation update subsection specs * CLDC-4140: update parsers post merge * CLDC-4140: update tests after field renumbering * CLDC-4140: update tests after field renumbering * CLDC-4140: update spacing in to_2026_row * CLDC-4140: update row parser spec field numbers * CLDC-4140: update row parser field types --------- Co-authored-by: Samuel Young <samuel.young@softwire.com>	3 weeks ago
Samuel Young	7821742b5c	CLDC-4151: Split Referral into three questions (#3150 ) * CLDC-4151: Note deprecations of existing referral questions * CLDC-4151: Add new cols to lettings log model * CLDC-4151: Add new referrals questions * CLDC-4151: Remove referral value check * CLDC-4151: Add new fields to bulk upload use new fields as presented in bulk upload * CLDC-4151: Update tests also add bulk upload file to test with * CLDC-4151: Ignore tests failing for later tickets * fixup! CLDC-4151: Add new referrals questions start q nums from 2026 * CLDC-4151: Add model tests for new questions * fixup! CLDC-4151: Add new fields to bulk upload export old_visible_id set MAX_COLUMNS correctly check .prp? for prp cols * fixup! CLDC-4151: Update tests clarify tests that may update * fixup! CLDC-4151: Add new fields to bulk upload handle the old_visible_id not existing use override org ID only for the output owning ID. it's only passed as eg ORG1 * CLDC-4151: Add new questions to log factory * fixup! CLDC-4151: Add new fields to bulk upload leave field_131 validations till later * CLDC-4188: Add LA flow splits the referral_register question and pages into two, as its not easy to have the answers be dependent if the owning organisation changes type, reset the referral register question. the other questions do not need to be reset as they are no longer routed to * CLDC-4188: Infer referral_register for renewals needs a new dependencies array for 2026 this handles inferring as well as clearing inferred answers if the prior answer changes * CLDC-4188: Add validation between prevten and referral_register block two other validations from previous years * CLDC-4188: Update tests * CLDC-4188: Ignore tests failing for future tickets * CLDC-4188: Update existing tests needed as should_reset_referral_register? calls a .find() * fixup! CLDC-4188: Add validation between prevten and referral_register use .prp? improve grammar * fixup! CLDC-4188: Infer referral_register for renewals extract dependencies to common list * fixup! CLDC-4188: Update tests use before and context blocks better * fixup! CLDC-4188: Update tests lint * fixup! CLDC-4188: Update tests remove unneeded referral_type set * CLDC-4189: Add PRP flow * CLDC-4189: Update validation between prevten and referral_register * CLDC-4189: Add tests * fixup! CLDC-4189: Add PRP flow fix typo Co-authored-by: Oscar Richardson <116292912+oscar-richardson-softwire@users.noreply.github.com> * fixup! CLDC-4189: Add tests fix typo * fixup! CLDC-4189: Add tests fix missing cases in page specs use a loop and before blocks for household validation tests * CLDC-4189: Ensure log is not classed as generan needs if prevten isnt answered * fixup! CLDC-4189: Add tests add cases for other internal transfer flow * CLDC-4151: Add final question numbers * fixup! CLDC-4189: Update validation between prevten and referral_register include new code 40 from CLDC-4149 * fixup! CLDC-4151: Add final question numbers reinstate fallback * CLDC-4190: Q84 Referral BU validations (#3155) * CLDC-4151: Note deprecations of existing referral questions * CLDC-4151: Add new cols to lettings log model * CLDC-4151: Add new referrals questions * CLDC-4151: Remove referral value check * CLDC-4151: Add new fields to bulk upload use new fields as presented in bulk upload * CLDC-4151: Update tests also add bulk upload file to test with * CLDC-4151: Ignore tests failing for later tickets * fixup! CLDC-4151: Add new referrals questions start q nums from 2026 * CLDC-4151: Add model tests for new questions * fixup! CLDC-4151: Add new fields to bulk upload export old_visible_id set MAX_COLUMNS correctly check .prp? for prp cols * fixup! CLDC-4151: Update tests clarify tests that may update * fixup! CLDC-4151: Add new fields to bulk upload handle the old_visible_id not existing use override org ID only for the output owning ID. it's only passed as eg ORG1 * CLDC-4151: Add new questions to log factory * fixup! CLDC-4151: Add new fields to bulk upload leave field_131 validations till later * CLDC-4188: Add LA flow splits the referral_register question and pages into two, as its not easy to have the answers be dependent if the owning organisation changes type, reset the referral register question. the other questions do not need to be reset as they are no longer routed to * CLDC-4188: Infer referral_register for renewals needs a new dependencies array for 2026 this handles inferring as well as clearing inferred answers if the prior answer changes * CLDC-4188: Add validation between prevten and referral_register block two other validations from previous years * CLDC-4188: Update tests * CLDC-4188: Ignore tests failing for future tickets * CLDC-4188: Update existing tests needed as should_reset_referral_register? calls a .find() * fixup! CLDC-4188: Add validation between prevten and referral_register use .prp? improve grammar * fixup! CLDC-4188: Infer referral_register for renewals extract dependencies to common list * fixup! CLDC-4188: Update tests use before and context blocks better * fixup! CLDC-4188: Update tests lint * fixup! CLDC-4188: Update tests remove unneeded referral_type set * CLDC-4189: Add PRP flow * CLDC-4189: Update validation between prevten and referral_register * CLDC-4189: Add tests * fixup! CLDC-4189: Add PRP flow fix typo Co-authored-by: Oscar Richardson <116292912+oscar-richardson-softwire@users.noreply.github.com> * fixup! CLDC-4189: Add tests fix typo * fixup! CLDC-4189: Add tests fix missing cases in page specs use a loop and before blocks for household validation tests * CLDC-4189: Ensure log is not classed as generan needs if prevten isnt answered * fixup! CLDC-4189: Add tests add cases for other internal transfer flow * CLDC-4190: Invalidate all referral fields if any are wrong * CLDC-4190: Remove other validations on referral fields * CLDC-4190: Ignore referral validation if BU is renewal * CLDC-4190: Add tests * fixup! CLDC-4190: Invalidate all referral fields if any are wrong add punctuation to comment name validation fields better * fixup! CLDC-4190: Add tests make test names clearer * CLDC-4190: Final field fixes --------- Co-authored-by: Oscar Richardson <116292912+oscar-richardson-softwire@users.noreply.github.com> * CLDC-4191: Q84 Referral CSV and XML export (#3157) * CLDC-4151: Note deprecations of existing referral questions * CLDC-4151: Add new cols to lettings log model * CLDC-4151: Add new referrals questions * CLDC-4151: Remove referral value check * CLDC-4151: Add new fields to bulk upload use new fields as presented in bulk upload * CLDC-4151: Update tests also add bulk upload file to test with * CLDC-4151: Ignore tests failing for later tickets * fixup! CLDC-4151: Add new referrals questions start q nums from 2026 * CLDC-4151: Add model tests for new questions * fixup! CLDC-4151: Add new fields to bulk upload export old_visible_id set MAX_COLUMNS correctly check .prp? for prp cols * fixup! CLDC-4151: Update tests clarify tests that may update * fixup! CLDC-4151: Add new fields to bulk upload handle the old_visible_id not existing use override org ID only for the output owning ID. it's only passed as eg ORG1 * CLDC-4151: Add new questions to log factory * fixup! CLDC-4151: Add new fields to bulk upload leave field_131 validations till later * CLDC-4188: Add LA flow splits the referral_register question and pages into two, as its not easy to have the answers be dependent if the owning organisation changes type, reset the referral register question. the other questions do not need to be reset as they are no longer routed to * CLDC-4188: Infer referral_register for renewals needs a new dependencies array for 2026 this handles inferring as well as clearing inferred answers if the prior answer changes * CLDC-4188: Add validation between prevten and referral_register block two other validations from previous years * CLDC-4188: Update tests * CLDC-4188: Ignore tests failing for future tickets * CLDC-4188: Update existing tests needed as should_reset_referral_register? calls a .find() * fixup! CLDC-4188: Add validation between prevten and referral_register use .prp? improve grammar * fixup! CLDC-4188: Infer referral_register for renewals extract dependencies to common list * fixup! CLDC-4188: Update tests use before and context blocks better * fixup! CLDC-4188: Update tests lint * fixup! CLDC-4188: Update tests remove unneeded referral_type set * CLDC-4189: Add PRP flow * CLDC-4189: Update validation between prevten and referral_register * CLDC-4189: Add tests * fixup! CLDC-4189: Add PRP flow fix typo Co-authored-by: Oscar Richardson <116292912+oscar-richardson-softwire@users.noreply.github.com> * fixup! CLDC-4189: Add tests fix typo * fixup! CLDC-4189: Add tests fix missing cases in page specs use a loop and before blocks for household validation tests * CLDC-4189: Ensure log is not classed as generan needs if prevten isnt answered * fixup! CLDC-4189: Add tests add cases for other internal transfer flow * CLDC-4190: Invalidate all referral fields if any are wrong * CLDC-4190: Remove other validations on referral fields * CLDC-4190: Ignore referral validation if BU is renewal * CLDC-4190: Add tests * fixup! CLDC-4190: Invalidate all referral fields if any are wrong add punctuation to comment name validation fields better * fixup! CLDC-4190: Add tests make test names clearer * CLDC-4191: Update CSV export tests the export happens by default, no changes needed here * CLDC-4191: Add new referral fields to 2026 exports remove old referral fields from 2026 exports ensure new referral fields don't show in pre 2026 exports * CLDC-4190: Final field fixes --------- Co-authored-by: Oscar Richardson <116292912+oscar-richardson-softwire@users.noreply.github.com> --------- Co-authored-by: Oscar Richardson <116292912+oscar-richardson-softwire@users.noreply.github.com>	4 weeks ago
Samuel Young	98c58cc19a	CLDC-4142: New question on gender same as sex - lettings (#3161 ) * CLDC-4142: Add new cols to database * CLDC-4142: Add questions to flow optimise household_characteristics as there's a lot of repeated code. should make the sections a bit easier to manage * CLDC-4142: Add conditional description question seems to largely just work without needing major tweaks * CLDC-4142: Ensure description if nil if not shown looks like there's no automated process for this. without this the description will be "" if not given * CLDC-4142: Add new question to log factory * CLDC-4142: Add new fields to BU parser update BU template file to match * CLDC-4142: Add new fields to BU file generator * CLDC-4142: Add new fields to xml export also improve xml export matcher to report differences * CLDC-4142: Add final question numbers * CLDC-4142: Add question specs * CLDC-4142: Add a PERSON_DETAILS override for gender_same_as_sex this ensures if details aren't known the field will be redacted * CLDC-4142: Update model tests * CLDC-4142: Update CSV export service tests * CLDC-4142: Update other tests * fixup! CLDC-4142: Update CSV export service tests update sales variable definitons too * fixup! CLDC-4142: Ensure description if nil if not shown update inferral method name and comment * CLDC-4142: Update CYA copy as requested cya for gender_same_as_sex 'no' instead of 'no, enter gender description' cya for gender_description has the label 'gender identity description' * CLDC-4142: Use label_from_value for custom CYA text this replaces @inferred_check_answers_value so it'll work in CSV export too	1 month ago
Nat Dean-Lewis	8c79d152b2	CLDC-4141: New sex registered at birth question for lettings logs (#3171 ) * CLDC-4177: Create new sex-at-birth question * CLDC-4177: Update database * CLDC-4177: Adds SAB form for the primary buyer. * CLDC-4177: Adds sex-at-birth question for other household members * CLCD-4177: Unit tests * CLDC-4177: Misc inclusions where sex is also used * CLDC-4177: Lint fixes * CLDC-4177: Do not include sexRAB in pre-2026 exports * CLDC-4177: Fixes bug with copy key in new page * CLDC-4177: Changes logic used to decide which fields to export * CLDC-4177: Linting fixes * CLDC-4177: Converts new field to lowercase * CLDC-4177: Satisfy bulk upload tests (not tested manually) * CLDC-4177: Rename new fields to lowercase * CLDC-4177: Rename field in bulk upload. * CLDC-4141: Update database * CLDC-4141: New question and page models * CLDC-4141: Adds the new sab questions to the form * CLDC-4141: Update locales * CLDC-4141: Export sab questions * CLDC-4141: change buyer -> lead tenant for lettings log sexrab1 * CLDC-4141: enable sexrab questions for lettings log persons 2-8 * CLDC-4141: update row parser * CLDC-4141: update csv service * Fix db casing * CLDC-4141: update schema (nonfunctional change) * CLDC-4141: update row parser expected fields * CLDC-4141: use correct lettings q * CLDC-4141: update page and question tests * CLDC-4141: update subsection test * CLDC-4141: update row parser test * CLDC-4141: linting * CLDC-4141: remove frozen string literals * CLDC-4177: remove frozen string literals * CLDC-4141: fix person casing typo * Delete redundant migration * CLDC-4141: revert fields count change * CLDC-4141: update row parser spec * CLDC-4141: revert export service change * Revert "CLDC-4141: revert export service change" This reverts commit `c43ff40006`. * CLDC-4177: Create new sex-at-birth question * CLDC-4177: Update database * CLDC-4177: Adds SAB form for the primary buyer. * CLDC-4177: Adds sex-at-birth question for other household members * CLCD-4177: Unit tests * CLDC-4177: Misc inclusions where sex is also used * CLDC-4177: Lint fixes * CLDC-4177: Do not include sexRAB in pre-2026 exports * CLDC-4177: Fixes bug with copy key in new page * CLDC-4177: Changes logic used to decide which fields to export * CLDC-4177: Linting fixes * CLDC-4177: Converts new field to lowercase * CLDC-4177: Satisfy bulk upload tests (not tested manually) * CLDC-4177: Rename new fields to lowercase * CLDC-4177: Rename field in bulk upload. * Fix db casing * CLDC-4177: remove frozen string literals * Delete redundant migration * Remove extraneous lines from schema.rb * Add new sexrab fields to the :completed trait in the Sales Log factory * CLDC-4141: partial csv download test update * CLDC-4141: update csv conversion rake task * CLDC-4141: untrack temp csv export files * CLDC-4141: update remaining csv tests * CLDC-4141: fix schema since latest changes * CLDC-4141: update schema * CLDC-4141: revert breaking export service changes * CLDC-4141: update var def spec * CLDC-4141: update var def spec * CLDC-4141: update var def spec * CLDC-4141: update BU test file * CLDC-4141: update validator specs * CLDC-4141: update csv_parser columns * CLDC-4141: linting * CLDC-4141: linting * CLDC-4141: linting * CLDC-4141: update person_n_present methods * CLDC-4141: remove redundant MAX_COLUMNS check * CLDC-4141: avoid sales changes in this PR, will leave to CLDC-4177 * CLDC-4141: use FIELDS const where can be shared * CLDC-4141: revert FIELDS sharing * CLDC-4141: update export services tests * CLDC-4141: include fields rather than omitting in lettings_log_export_service.rb * CLDC-4141: share default 2026 field count * CLDC-4141: cleanup export field filtering * CLDC-4141: simplify export year filtering * CLDC-4141: use FIELDS to check max_columns * CLDC-4141: update 2026_27 manual test fixture * CLDC-4177: remove redundant inferred CYA vals * CLDC-4141: remove redundant inferred CYA vals * CLDC-4177: update bu files * CLDC-4177: update sales log export service * CLDC-4141: remove redundant field filtering * CLDC-4141: reinstate old comment * CLDC-4141: reinstate old comment * CLDC-4141: rename var def file * CLDC-4141: update variable definitions tests * CLDC-4141: update schema * CLDC-4177: update schema * CLDC-4177: linting * CLDC-4177: linting * CLDC-4141: update csv fixtures * CLDC-4177: update sales log export constants * CLDC-4177: update factories * CLDC-4177: lint * CLDC-4177: csv parser updates * CLDC-4177: bu field updates * CLDC-4177: update var def spec * CLDC-4141: Add final question numbers (CLDC-4202) * CLDC-4141: make to_2026_row explicit * CLDC-4177: make to_2026_row explicit * CLDC-4177: use shared field count * CLDC-4177: refactor to remove max cols going forwards * CLDC-4141: update schema * CLDC-4141: update schema * CLDC-4141: update schema * CLDC-4141: update schema * CLDC-4141: remove X from factory options * CLDC-4177: update log var defs * CLDC-4177: use correct shared field count * CLDC-4177: remove trailing commas * CLDC-4177: update log var defs * CLDC-4177: update bulk upload csv example * CLDC-4141: update bulk upload csv example * CLDC-4177: update bulk upload csv example * CLDC-4177: remove sexrab from expected sales export for now * CLDC-4141: correct csv service expectations * CLDC-4141: don't sample X for sexrab --------- Co-authored-by: Katherine Langford <katherine.langford@softwire.com> Co-authored-by: oscar-richardson-softwire <oscar.richardson@softwire.com> Co-authored-by: Samuel Young <samuel.young@softwire.com>	1 month ago
Katherine Langford	d2fff16808	CLDC-4177: New Sex At Birth question for sales logs (#3144 ) * CLDC-4177: Create new sex-at-birth question * CLDC-4177: Update database * CLDC-4177: Adds SAB form for the primary buyer. * CLDC-4177: Adds sex-at-birth question for other household members * CLCD-4177: Unit tests * CLDC-4177: Misc inclusions where sex is also used * CLDC-4177: Lint fixes * CLDC-4177: Do not include sexRAB in pre-2026 exports * CLDC-4177: Fixes bug with copy key in new page * CLDC-4177: Changes logic used to decide which fields to export * CLDC-4177: Linting fixes * CLDC-4177: Converts new field to lowercase * CLDC-4177: Satisfy bulk upload tests (not tested manually) * CLDC-4177: Rename new fields to lowercase * CLDC-4177: Rename field in bulk upload. * Fix db casing * CLDC-4177: remove frozen string literals * Delete redundant migration * Remove extraneous lines from schema.rb * Add new sexrab fields to the :completed trait in the Sales Log factory * CLDC-4177: remove redundant inferred CYA vals * CLDC-4177: update bu files * CLDC-4177: update sales log export service * CLDC-4177: update schema * CLDC-4177: linting * CLDC-4177: update sales log export constants * CLDC-4177: update factories * CLDC-4177: lint * CLDC-4177: csv parser updates * CLDC-4177: bu field updates * CLDC-4177: update var def spec * CLDC-4177: make to_2026_row explicit * CLDC-4177: use shared field count * CLDC-4177: refactor to remove max cols going forwards * CLDC-4141: update schema * CLDC-4141: update schema * CLDC-4141: update schema * CLDC-4141: remove X from factory options * CLDC-4177: update log var defs * CLDC-4177: use correct shared field count * CLDC-4177: remove trailing commas * CLDC-4177: update bulk upload csv example * CLDC-4177: update bulk upload csv example * CLDC-4177: remove sexrab from expected sales export for now --------- Co-authored-by: oscar-richardson-softwire <oscar.richardson@softwire.com> Co-authored-by: Nat Dean-Lewis <nat.dean-lewis@softwire.com>	1 month ago
Samuel Young	a513984440	CLDC-4162: Set up Bulk Uploads for 2026 (#3134 ) * CLDC-4162: Add parsers for 2026 * CLDC-4162: Integrate 2026 parsers into codebase * CLDC-4162: Add tests * CLDC-4162: Update tests for existing years remove year dependencies where possible * CLDC-4162: Add example 2026 bulk upload files * CLDC-4162: Update BU test file generate buttons * fixup! CLDC-4162: Update BU test file generate buttons we only need the 14 days validation when the future forms feature toggle is off, otherwise any date can be input * fixup! CLDC-4162: Integrate 2026 parsers into codebase improve TODOs * fixup! CLDC-4162: Add parsers for 2026 add todos for unknown info * fixup! CLDC-4162: Update tests for existing years properly mark year specific tests * fixup! CLDC-4162: Update tests for existing years use 25 over 2025 * fixup! CLDC-4162 Update BU test file generate buttons --------- Co-authored-by: carolynbarker <8038496+carolynbarker@users.noreply.github.com>	2 months ago
carolynbarker	b02bfcb7cd	CLDC-3939 update tests for new FY (#3010 ) * test tests in April * update tests where 2014 is assumed * update test where time moving is assumed * update rake task tests that only affect 2024 logs * update another 2024 assumption * handle null period in validation * give pages different ids in SharedOwnershipInitialPurchase and SharedOwnershipStaircasingTransaction sections * refactor to remove unnecessary method * delete some old rake tasks and tests * CLDC-3899 remove flaky date comparison from tests * CLDC-3899 fix flaky test occasionally picking bad input * fix test year setup * add relat mapping for log_to_csv * temp disable fail-fast for ease of testing * specify 2024 in 2024-specific test * Revert "temp disable fail-fast for ease of testing" This reverts commit `6993d29866`. * Revert "test tests in April" This reverts commit `f5635dba11`. * pass test in 2024 and 2025 * fix test to 2024 * temp disable fail-fast again * Revert "temp disable fail-fast again" This reverts commit `55d3e1c7d8`. --------- Co-authored-by: Carolyn <carolyn.barker@softwire.com>	12 months ago
kosiakkatrina	71c3f41bf9	CLDC-3803 Add lettings 25/26 BU (#2939 ) * Copy row and csv parser for 2025 lettings * Add 2025 BU translations file * Update row parser field number * Add "apecified accommodation" to rent type mapping * Update csv parser * Add prepare your file and connect the csv parser to log creator and validator * Update test file creation * Refactor some csv methods, add person relationship mapping * Update row parser specs * Further refactor some test helper methods * Merge update	1 year ago
carolynbarker	e5d10e219f	CLDC-3802 sales bulk upload update 2025 (#2945 ) * add 2025 sales bulk upload parser * lint and fix typos * fix typos, field numbers, staircasing tests * order field_11 values * test log creator selects correct year's parser * fix log to csv helper field order * update factory so test file fully succeeds * add 2025 BU test file method * apply new csv syntax * lint * lint * CLDC-3893 update property information field order * commonise prepare your file page * also update prep file page for lettings * CLDC-3893 update test * lint * don't error on blank discount if not RTB --------- Co-authored-by: Carolyn <carolyn.barker@softwire.com>	1 year ago
carolynbarker	f268357616	refactor repetitive sales csv methods (#2954 ) * refactor repetitive sales csv methods * remove 2025 test until 2025 functionality is actually merged * fix syntax typo * fix incorrect method * fix indentation --------- Co-authored-by: Carolyn <carolyn.barker@softwire.com>	1 year ago
kosiakkatrina	830adc0dd7	Update test file generation (#2868 ) * Extract test file generation * Remove secondary * Make link shorter * Move log_to_csv classes to helpers	1 year ago

12 Commits (e01ae69f21144ee2cd9d22cc4066a7ba85860080)