Skip to content

Resolve "Cannot bulk clear or update when a table contains a error formula field which has a db column type not compatible with text"

Steps to reproduce

  1. Create a single select field/link row field/lookup field
  2. Reference the field with a formula field
  3. Delete the field breaking the formula field
  4. Try to bulk copy and paste or clear
  5. See the error

The formula field goes into an error state when the field it depends on is deleted. When a formula is in an error state internally we use a TextField model field to represent on the model. But we don't actually alter the underlying db column to become a text field (we don't want deleting a field to cause cascading ALTER TABLE in every single connected table!).

Normally this is OK as we can just return Value(None) for all error formulas in normal UPDATEs and INSERTS. However Djangos bulk_update function is special compared to every other one and works differently. It explicitly cast's all formula expressions to the model field's type, which Django doesn't do elsewhere. Which for an error field is a TextField. So it then tries to run the SQL SET jsonb_column=CASE WHEN id=1 CAST(NULL AS TEXT) ... which is invalid SQL as the underlying column is still JSONB etc.

Possible Fixes

1. Just don't bulk update error formulas

This is the approach I've gone with initially. We only ever bulk_update generated table models in this one place, and so I think a nice and simple way of preventing this crash is just by not including these error formulas the the bulk_update statement. This means we never hit the problem of Django trying to cast the error expression to TEXT for a JSONB column type.

2. Fundamentally change the Formula error type to remember what type it used to be

Right now when a formula breaks and is put into the error type, we override its old type information. We however do not ALTER it's column in the database but just leave it as it is and/or NULL all of it's values. Instead we could change this so we do somehow remember the pre-error underlying column type and when we construct a model field for a formula in an error state we construct it not always as a TextField as we do now, but a model_field which actually matches the column in the DB. This is a much more complex change (we'd need a migration that checked the actual column types in the tables to reverse engineer what the pre-error column was etc.)

Merge Request Checklist

Closes #1084 (closed)

Edited by Nigel Gott

Merge request reports