submit-social-housing-lettings-and-sales-data

1894 Commits

272 Branches

306 Tags

86 MiB

Tree: 856524ae21

Commit Graph

Author	SHA1	Message	Date
Phil Lee	0009d535fe	CLDC-1896 Bulk upload line endings (#1208 ) * add class to create logs from bulk upload * create logs when processing bulk uploads * remove bulk_upload_id from csv output * create bulk upload logs only if all valid - this will be changed later to allow for partial logs - and only to create logs when a threshold has been met * add method to blank invalid non setup fields * bulk upload log creation blanks invalid fields * fix incorrect logic for bulk upload renewal * fix linting * bulk upload log creation fail logs to sentry * fix bulk upload line ending parsing * extract bulk uploading csv parsing to class * use csv parser in log creator * change handle line endings mechanism - we now strip all windows line endings for unix based line endings - this normalises things making it simpler	3 years ago
Phil Lee	90480d1af5	CLDC-1779 Bulk upload lettings validation (#1148 ) * able to view lettings bulk upload errors * fix linting * call service correctly in test * add bulk upload sales questions mapping * appease linter * bulk upload error shows correct question - depending on log type it will show relevant question for the field concerned * improve namespacing of classes * add job to process bulk uploads * move validation from parser to model * add validations for field_1 * add validation for field_4 * pending tests for field_4 * convert field_mapping to array of hashes * validate nulls based on form question * actually load forms when toggling between forms * validate null for startdate * row parser has access to bulk upload * csv upload validates first form section * add postcode validation * Refactor error mappings for row parser * Add unittype question * Fix null error setting and add builtype * add wchair to bulk upload * Add beds to bulk upload * Add joint to bulk upload * Add startertenancy to the bulk upload * Add tenancy for bulk upload * Add declaration to the bulk upload * Add age1 and age1_known to bulk upload * add ages to bulk upload * add sex1 to bulk upload * add ethnic_group and ethnic to bulk upload * add national to bulk upload * add ecstat1 to bulk upload * add military related fields to bulk upload * add preg_occ to bulk upload * add housingneeds to bulk upload * add illness to bulk upload * add layear to bulk upload * add waityear to bulk upload * add reason to bulk upload * add prevten to bulk upload * add homeless to bulk upload * add previous postcode to bulk upload * add reasonable preferences to bulk upload * add allocations system to bulk upload * add referral to bulk upload * add net_income_known to bulk upload * add hb to bulk upload * add benefits to bulk upload * add rent fields to bulk upload * add hhmemb to bulk upload * use 2022 csv fixtures for bulk upload * fix renewal mapping for bulk upload * placeholder test for bulk upload validation * fix bulk upload mapping for homeless field * fix leftreg mapping for bulk upload * fix user associations in bulk upload tests * add gender fields for bulk upload * add ecstatN fields to bulk upload * add #relatN fields to bulk upload * extract old_visible_id in factory to trait * map net_income_known correctly for bulk upload * fix income bugs for bulk upload * add unitletas to bulk upload * add #rsnvac to bulk upload * add #sheltered to bulk upload * add illness fields to bulk upload * add #irproduct_other to bulk upload * infer renewal from rsnvac for bulk upload * add #tenancyother to bulk upload * add #tenancylength to bulk upload * bulk upload earnings accepts pennies but rounds * add #reasonother to bulk upload * fix mapping of #ppcodenk for bulk upload * add #household_charge to bulk upload * add #chcharge to bulk upload * add #tcharge to bulk upload * add #supcharg to bulk upload * add pscharge to bulk upload * add #scharge to bulk upload * use case statement for bulk upload allocation * add offered to bulk upload * add propcode to bulk upload * add major repair fields to bulk upload * add #voiddate to bulk upload * support YY year format for bulk upload * test postcode strips whitespace for bulk upload * add #la to bulk upload * add previous la to bulk upload * fix failing test * remove duplicate line from rebase * add first time social housing to bulk upload * make methods private * fix field_4 validation for bulk upload - the null check was inverted by mistake Co-authored-by: Kat <katrina@kosiak.co.uk>	3 years ago
Phil Lee	197e2ae4cb	[1778] Bulk upload errors (#1091 ) * lockdown bulk upload routes * able to view lettings bulk upload errors * add error count to bulk upload results * coverage for bulk upload filename on results * group bulk upload errors by row on results * able to view bulk upload sales results * scope lettings and sales bulk upload results * fix linting * call service correctly in test * add bulk upload sales questions mapping * appease linter * bulk upload error shows correct question - depending on log type it will show relevant question for the field concerned * use local disk for bulk upload for dev env - this saves the need to connect to S3 to play with bulk upload in dev environment * improve namespacing of classes * add job to process bulk uploads * use local disk storage for dev file upload * fix test * use inline active job queue_adapter for dev * use test active job queue adapter for test env * remove rubocop violation * delete bulk upload from disk after processing * populate errors with cell, row + metadata * update error message with something meaningful * shim in sales validator * able to parse sales bulk uploads * change migration to add purchase_code to errors * bulk upload error component renders purchaser code - when a sales log * populate purchaser_code for bulk upload errors - when log type is sales * remove superfluous private method	3 years ago

Author

SHA1

Message

Date

Phil Lee

0009d535fe

CLDC-1896 Bulk upload line endings (#1208 )

* add class to create logs from bulk upload

* create logs when processing bulk uploads

* remove bulk_upload_id from csv output

* create bulk upload logs only if all valid

- this will be changed later to allow for partial logs
- and only to create logs when a threshold has been met

* add method to blank invalid non setup fields

* bulk upload log creation blanks invalid fields

* fix incorrect logic for bulk upload renewal

* fix linting

* bulk upload log creation fail logs to sentry

* fix bulk upload line ending parsing

* extract bulk uploading csv parsing to class

* use csv parser in log creator

* change handle line endings mechanism

- we now strip all windows line endings for unix based line endings
- this normalises things making it simpler

3 years ago

Phil Lee

90480d1af5

CLDC-1779 Bulk upload lettings validation (#1148 )

* able to view lettings bulk upload errors

* fix linting

* call service correctly in test

* add bulk upload sales questions mapping

* appease linter

* bulk upload error shows correct question

- depending on log type it will show relevant question for the field
  concerned

* improve namespacing of classes

* add job to process bulk uploads

* move validation from parser to model

* add validations for field_1

* add validation for field_4

* pending tests for field_4

* convert field_mapping to array of hashes

* validate nulls based on form question

* actually load forms when toggling between forms

* validate null for startdate

* row parser has access to bulk upload

* csv upload validates first form section

* add postcode validation

* Refactor error mappings for row parser

* Add unittype question

* Fix null error setting and add builtype

* add wchair to bulk upload

* Add beds to bulk upload

* Add joint to bulk upload

* Add startertenancy to the bulk upload

* Add tenancy for bulk upload

* Add declaration to the bulk upload

* Add age1 and age1_known to bulk upload

* add ages to bulk upload

* add sex1 to bulk upload

* add ethnic_group and ethnic to bulk upload

* add national to bulk upload

* add ecstat1 to bulk upload

* add military related fields to bulk upload

* add preg_occ to bulk upload

* add housingneeds to bulk upload

* add illness to bulk upload

* add layear to bulk upload

* add waityear to bulk upload

* add reason to bulk upload

* add prevten to bulk upload

* add homeless to bulk upload

* add previous postcode to bulk upload

* add reasonable preferences to bulk upload

* add allocations system to bulk upload

* add referral to bulk upload

* add net_income_known to bulk upload

* add hb to bulk upload

* add benefits to bulk upload

* add rent fields to bulk upload

* add hhmemb to bulk upload

* use 2022 csv fixtures for bulk upload

* fix renewal mapping for bulk upload

* placeholder test for bulk upload validation

* fix bulk upload mapping for homeless field

* fix leftreg mapping for bulk upload

* fix user associations in bulk upload tests

* add gender fields for bulk upload

* add ecstatN fields to bulk upload

* add #relatN fields to bulk upload

* extract old_visible_id in factory to trait

* map net_income_known correctly for bulk upload

* fix income bugs for bulk upload

* add unitletas to bulk upload

* add #rsnvac to bulk upload

* add #sheltered to bulk upload

* add illness fields to bulk upload

* add #irproduct_other to bulk upload

* infer renewal from rsnvac for bulk upload

* add #tenancyother to bulk upload

* add #tenancylength to bulk upload

* bulk upload earnings accepts pennies but rounds

* add #reasonother to bulk upload

* fix mapping of #ppcodenk for bulk upload

* add #household_charge to bulk upload

* add #chcharge to bulk upload

* add #tcharge to bulk upload

* add #supcharg to bulk upload

* add pscharge to bulk upload

* add #scharge to bulk upload

* use case statement for bulk upload allocation

* add offered to bulk upload

* add propcode to bulk upload

* add major repair fields to bulk upload

* add #voiddate to bulk upload

* support YY year format for bulk upload

* test postcode strips whitespace for bulk upload

* add #la to bulk upload

* add previous la to bulk upload

* fix failing test

* remove duplicate line from rebase

* add first time social housing to bulk upload

* make methods private

* fix field_4 validation for bulk upload

- the null check was inverted by mistake

Co-authored-by: Kat <katrina@kosiak.co.uk>

3 years ago

Phil Lee

197e2ae4cb

[1778] Bulk upload errors (#1091 )

* lockdown bulk upload routes

* able to view lettings bulk upload errors

* add error count to bulk upload results

* coverage for bulk upload filename on results

* group bulk upload errors by row on results

* able to view bulk upload sales results

* scope lettings and sales bulk upload results

* fix linting

* call service correctly in test

* add bulk upload sales questions mapping

* appease linter

* bulk upload error shows correct question

- depending on log type it will show relevant question for the field
  concerned

* use local disk for bulk upload for dev env

- this saves the need to connect to S3 to play with bulk upload in dev
  environment

* improve namespacing of classes

* add job to process bulk uploads

* use local disk storage for dev file upload

* fix test

* use inline active job queue_adapter for dev

* use test active job queue adapter for test env

* remove rubocop violation

* delete bulk upload from disk after processing

* populate errors with cell, row + metadata

* update error message with something meaningful

* shim in sales validator

* able to parse sales bulk uploads

* change migration to add purchase_code to errors

* bulk upload error component renders purchaser code

- when a sales log

* populate purchaser_code for bulk upload errors

- when log type is sales

* remove superfluous private method

3 years ago

3 Commits (856524ae2183fc2577587c4c30a33b0ac8853be2)