Commit 16828d5c authored by Andrew Dunstan's avatar Andrew Dunstan

Fast ALTER TABLE ADD COLUMN with a non-NULL default

Currently adding a column to a table with a non-NULL default results in
a rewrite of the table. For large tables this can be both expensive and
disruptive. This patch removes the need for the rewrite as long as the
default value is not volatile. The default expression is evaluated at
the time of the ALTER TABLE and the result stored in a new column
(attmissingval) in pg_attribute, and a new column (atthasmissing) is set
to true. Any existing row when fetched will be supplied with the
attmissingval. New rows will have the supplied value or the default and
so will never need the attmissingval.

Any time the table is rewritten all the atthasmissing and attmissingval
settings for the attributes are cleared, as they are no longer needed.

The most visible code change from this is in heap_attisnull, which
acquires a third TupleDesc argument, allowing it to detect a missing
value if there is one. In many cases where it is known that there will
not be any (e.g.  catalog relations) NULL can be passed for this
argument.

Andrew Dunstan, heavily modified from an original patch from Serge
Rielau.
Reviewed by Tom Lane, Andres Freund, Tomas Vondra and David Rowley.

Discussion: https://postgr.es/m/31e2e921-7002-4c27-59f5-51f08404c858@2ndQuadrant.com
parent ef1978d6
......@@ -1149,6 +1149,19 @@
</entry>
</row>
<row>
<entry><structfield>atthasmissing</structfield></entry>
<entry><type>bool</type></entry>
<entry></entry>
<entry>
This column has a value which is used where the column is entirely
missing from the row, as happens when a column is added with a
non-volatile <literal>DEFAULT</literal> value after the row is created.
The actual value used is stored in the
<structfield>attmissingval</structfield> column.
</entry>
</row>
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
......@@ -1229,6 +1242,20 @@
</entry>
</row>
<row>
<entry><structfield>attmissingval</structfield></entry>
<entry><type>anyarray</type></entry>
<entry></entry>
<entry>
This column has a one element array containing the value used when the
column is entirely missing from the row, as happens when the column is
added with a non-volatile <literal>DEFAULT</literal> value after the
row is created. The value is only used when
<structfield>atthasmissing</structfield> is true. If there is no value
the column is null.
</entry>
</row>
</tbody>
</tgroup>
</table>
......
......@@ -1184,26 +1184,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
When a column is added with <literal>ADD COLUMN</literal>, all existing
rows in the table are initialized with the column's default value
(NULL if no <literal>DEFAULT</literal> clause is specified).
If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
change and does not require any immediate update of the table's data;
the added NULL values are supplied on readout, instead.
When a column is added with <literal>ADD COLUMN</literal> and a
non-volatile <literal>DEFAULT</literal> is specified, the default is
evaluated at the time of the statement and the result stored in the
table's metadata. That value will be used for the column for all existing
rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
neither case is a rewrite of the table required.
</para>
<para>
Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
an existing column will require the entire table and its indexes to be
rewritten. As an exception when changing the type of an existing column,
if the <literal>USING</literal> clause does not change the column
contents and the old type is either binary coercible to the new type or
an unconstrained domain over the new type, a table rewrite is not needed;
but any indexes on the affected columns must still be rebuilt. Adding or
removing a system <literal>oid</literal> column also requires rewriting the entire
table. Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
Adding a column with a volatile <literal>DEFAULT</literal> or
changing the type of an existing column will require the entire table and
its indexes to be rewritten. As an exception, when changing the type of an
existing column, if the <literal>USING</literal> clause does not change
the column contents and the old type is either binary coercible to the new
type or an unconstrained domain over the new type, a table rewrite is not
needed; but any indexes on the affected columns must still be rebuilt.
Adding or removing a system <literal>oid</literal> column also requires
rewriting the entire table. Table and/or index rebuilds may take a
significant amount of time for a large table; and will temporarily require
as much as double the disk space.
</para>
<para>
......
This diff is collapsed.
......@@ -21,12 +21,14 @@
#include "access/hash.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
......@@ -129,6 +131,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
att->attnotnull = false;
att->atthasdef = false;
att->atthasmissing = false;
att->attidentity = '\0';
}
......@@ -176,6 +179,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
if (constr->missing)
{
cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
for (i = tupdesc->natts - 1; i >= 0; i--)
{
if (constr->missing[i].ammissingPresent)
{
Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
attr->attbyval,
attr->attlen);
}
}
}
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
......@@ -227,6 +247,7 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
att->attnotnull = false;
att->atthasdef = false;
att->atthasmissing = false;
att->attidentity = '\0';
}
dst->constr = NULL;
......@@ -279,6 +300,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
/* since we're not copying constraints or defaults, clear these */
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
}
......@@ -309,6 +331,18 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
if (tupdesc->constr->missing)
{
AttrMissing *attrmiss = tupdesc->constr->missing;
for (i = tupdesc->natts - 1; i >= 0; i--)
{
if (attrmiss[i].ammissingPresent
&& !TupleDescAttr(tupdesc, i)->attbyval)
pfree(DatumGetPointer(attrmiss[i].ammissing));
}
pfree(attrmiss);
}
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
......@@ -469,6 +503,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
if (constr1->missing)
{
if (!constr2->missing)
return false;
for (i = 0; i < tupdesc1->natts; i++)
{
AttrMissing *missval1 = constr1->missing + i;
AttrMissing *missval2 = constr2->missing + i;
if (missval1->ammissingPresent != missval2->ammissingPresent)
return false;
if (missval1->ammissingPresent)
{
Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, i);
if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
missatt1->attbyval, missatt1->attlen))
return false;
}
}
}
else if (constr2->missing)
return false;
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
......@@ -584,6 +641,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
......@@ -642,6 +700,7 @@ TupleDescInitBuiltinEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
......@@ -797,6 +856,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
constr->missing = NULL;
constr->num_defval = 0;
constr->check = NULL;
constr->num_check = 0;
......
......@@ -4588,7 +4588,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
......
......@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
#include "optimizer/var.h"
#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
......@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
......@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
false, 'p', 's', true, false, '\0', false, true, 0
false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
true, 'p', 'i', true, false, '\0', false, true, 0
true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
true, 'p', 'i', true, false, '\0', false, true, 0
true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
true, 'p', 'i', true, false, '\0', false, true, 0
true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
true, 'p', 'i', true, false, '\0', false, true, 0
true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
true, 'p', 'i', true, false, '\0', false, true, 0
true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
......@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
true, 'p', 'i', true, false, '\0', false, true, 0
true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
......@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
......@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
......@@ -1925,14 +1931,91 @@ heap_drop_with_catalog(Oid relid)
}
/*
* RelationClearMissing
*
* Set atthasmissing and attmissingval to false/null for all attributes
* where they are currently set. This can be safely and usefully done if
* the table is rewritten (e.g. by VACUUM FULL or CLUSTER) where we know there
* are no rows left with less than a full complement of attributes.
*
* The caller must have an AccessExclusive lock on the relation.
*/
void
RelationClearMissing(Relation rel)
{
Relation attr_rel;
Oid relid = RelationGetRelid(rel);
int natts = RelationGetNumberOfAttributes(rel);
int attnum;
Datum repl_val[Natts_pg_attribute];
bool repl_null[Natts_pg_attribute];
bool repl_repl[Natts_pg_attribute];
Form_pg_attribute attrtuple;
HeapTuple tuple,
newtuple;
memset(repl_val, 0, sizeof(repl_val));
memset(repl_null, false, sizeof(repl_null));
memset(repl_repl, false, sizeof(repl_repl));
repl_val[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(false);
repl_null[Anum_pg_attribute_attmissingval - 1] = true;
repl_repl[Anum_pg_attribute_atthasmissing - 1] = true;
repl_repl[Anum_pg_attribute_attmissingval - 1] = true;
/* Get a lock on pg_attribute */
attr_rel = heap_open(AttributeRelationId, RowExclusiveLock);
/* process each non-system attribute, including any dropped columns */
for (attnum = 1; attnum <= natts; attnum++)
{
tuple = SearchSysCache2(ATTNUM,
ObjectIdGetDatum(relid),
Int16GetDatum(attnum));
if (!HeapTupleIsValid(tuple)) /* shouldn't happen */
elog(ERROR, "cache lookup failed for attribute %d of relation %u",
attnum, relid);
attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
/* ignore any where atthasmissing is not true */
if (attrtuple->atthasmissing)
{
newtuple = heap_modify_tuple(tuple, RelationGetDescr(attr_rel),
repl_val, repl_null, repl_repl);
CatalogTupleUpdate(attr_rel, &newtuple->t_self, newtuple);
heap_freetuple(newtuple);
}
ReleaseSysCache(tuple);
}
/*
* Our update of the pg_attribute rows will force a relcache rebuild, so
* there's nothing else to do here.
*/
heap_close(attr_rel, RowExclusiveLock);
}
/*
* Store a default expression for column attnum of relation rel.
*
* Returns the OID of the new pg_attrdef tuple.
*
* add_column_mode must be true if we are storing the default for a new
* attribute, and false if it's for an already existing attribute. The reason
* for this is that the missing value must never be updated after it is set,
* which can only be when a column is added to the table. Otherwise we would
* in effect be changing existing tuples.
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
Node *expr, bool is_internal)
Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
......@@ -2000,8 +2083,69 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
attStruct->atthasdef = true;
Form_pg_attribute defAttStruct;
ExprState *exprState;
Expr *expr2 = (Expr *) expr;
EState *estate = NULL;
ExprContext *econtext;
Datum valuesAtt[Natts_pg_attribute];
bool nullsAtt[Natts_pg_attribute];
bool replacesAtt[Natts_pg_attribute];
Datum missingval = (Datum) 0;
bool missingIsNull = true;
MemSet(valuesAtt, 0, sizeof(valuesAtt));
MemSet(nullsAtt, false, sizeof(nullsAtt));
MemSet(replacesAtt, false, sizeof(replacesAtt));
valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
if (add_column_mode)
{
expr2 = expression_planner(expr2);
estate = CreateExecutorState();
exprState = ExecPrepareExpr(expr2, estate);
econtext = GetPerTupleExprContext(estate);
missingval = ExecEvalExpr(exprState, econtext,
&missingIsNull);
FreeExecutorState(estate);
defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
if (missingIsNull)
{
/* if the default evaluates to NULL, just store a NULL array */
missingval = (Datum) 0;
}
else
{
/* otherwise make a one-element array of the value */
missingval = PointerGetDatum(
construct_array(&missingval,
1,
defAttStruct->atttypid,
defAttStruct->attlen,
defAttStruct->attbyval,
defAttStruct->attalign));
}
valuesAtt[Anum_pg_attribute_atthasmissing - 1] = !missingIsNull;
replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
}
atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
valuesAtt, nullsAtt, replacesAtt);
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
if (!missingIsNull)
pfree(DatumGetPointer(missingval));
}
heap_close(attrrel, RowExclusiveLock);
heap_freetuple(atttup);
......@@ -2185,7 +2329,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
is_internal);
is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
......@@ -2301,7 +2445,12 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
/* If the DEFAULT is volatile we cannot use a missing value */
if (colDef->missingMode && contain_volatile_functions((Node *) expr))
colDef->missingMode = false;
defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
......
......@@ -372,6 +372,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
......@@ -1655,7 +1656,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
......
......@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
......@@ -1669,6 +1669,16 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
}
relation_close(newrel, NoLock);
}
/* if it's not a catalog table, clear any missing attribute settings */
if (!is_system_catalog)
{
Relation newrel;
newrel = heap_open(OIDOldHeap, NoLock);
RelationClearMissing(newrel);
relation_close(newrel, NoLock);
}
}
......
......@@ -2252,7 +2252,7 @@ ExecuteCallStmt(CallStmt *stmt, ParamListInfo params, bool atomic, DestReceiver
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
......
......@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
heap_attisnull(tuple, Anum_pg_index_indexprs) &&
if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
......
......@@ -714,6 +714,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
......@@ -4682,7 +4683,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
if (heap_attisnull(tuple, attn + 1))
if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
......@@ -4785,7 +4786,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
......@@ -5404,6 +5405,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
......@@ -5448,6 +5450,13 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
/*
* Attempt to skip a complete table rewrite by storing the specified
* DEFAULT value outside of the heap. This may be disabled inside
* AddRelationNewConstraints if the optimization cannot be applied.
*/
rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
* _list_ of defaults, but we just do one.
......@@ -5457,6 +5466,13 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
/*
* Did the request for a missing value work? If not we'll have to do
* a rewrite
*/
if (!rawEnt->missingMode)
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
/*
......@@ -5502,6 +5518,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
/* must do a rewrite for identity columns */
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
......@@ -5537,16 +5556,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
/*
* If the new column is NOT NULL, tell Phase 3 it needs to test that.
* (Note we don't do this for an OID column. OID will be marked not
* null, but since it's filled specially, there's no need to test
* anything.)
*/
tab->new_notnull |= colDef->is_not_null;
if (DomainHasConstraints(typeOid))
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
{
/*
* If the new column is NOT NULL, and there is no missing value,
* tell Phase 3 it needs to test that. (Note we don't do this for
* an OID column. OID will be marked not null, but since it's
* filled specially, there's no need to test anything.)
*/
tab->new_notnull |= colDef->is_not_null;
}
}
/*
......@@ -6022,6 +6046,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
......@@ -8109,8 +8134,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
heap_attisnull(indexTuple, Anum_pg_index_indexprs))
heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
......@@ -9516,7 +9541,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
StoreAttrDefault(rel, attnum, defaultexpr, true);
StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
......
......@@ -2397,7 +2397,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
if (heap_attisnull(tuple, attnum))
if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
......
......@@ -2505,7 +2505,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
if (heap_attisnull(&tmptup, att))
if (heap_attisnull(&tmptup, att, tupDesc))
{