Now requires a RawStmt rather than any old Node. Methods to parse DDL
into a list of Nodes now return a list of RawStmts; the old Nodes we
expect are available as one of their fields. ParseTreeRawStmt is used
in PostgreSQL 10 to obtain the RawStmt itself; the behavior of the old
ParseTreeNode is preserved and it always returns a simple Node.
getOwnedSequences can now filter owned sequences by a specified column.
Passing InvalidAttrNumber preserves the original behavior of returning
all sequences owned by a specified relation.
PostgreSQL 10 removes the ability to directly call CatalogUpdateIndexes
in favor of new CatalogTupleUpdate and CatalogTupleInsert calls, which
do this for us.
Certain PostgreSQL 10 changes require modifications to our includes:
* pg_sequence's interace has been refactored into its own file
* textToQualifiedNameList is now in utils/varlena.h
* pg_getnameinfo_all is now in common/ip.h
* stringToQualifiedNameList is now in utils/regproc.h
WaitLatchOrSocket now accepts a "wait_event_info" argument for use by
pg_stat. Extensions appear to mostly just pass "PG_WAIT_EXTENSION", so
that's what I've done.
PostgreSQL 10's ProcessUtility now requires a PlannedStmt with its
utilityStmt field set to what we used to pass directly.
In addition, it receives a QueryEnvironment reference, altogether new
but apparently OK to NULL out for now.
This adds a wrapper to adapt to the new style and fixes all callers.
- Use native postgres function for composite key btree functions
- Move explain tests to multi_explain.sql (get rid of .out _0.out files)
- Get rid of input/output files for multi_subquery.sql by moving table creations
- Update some comments
With this commit we start to register InvalidateDistRelationCacheCallback
function as cache invalidation callback function before version checks
because during version checks we use cache to look up relation ids of some
relations like pg_dist_relation or pg_dist_partition_logical_relid_index
and we want to know about cache invalidation before accessing them.
During version update, we indirectly calld CheckInstalledVersion via
ChackCitusVersions. This obviously fails because during version update it is
expected to have version mismatch between installed version and binary version.
Thus, we remove that ChackCitusVersions. We now only call ChackAvailableVersion.
Before this commit, we were erroring out at almost all queries if there is a
version mismatch. With this commit, we started to error out only requested
operation touches distributed tables.
Normally we would need to use distributed cache to understand whether a table
is distributed or not. However, it is not safe to read our metadata tables when
there is a version mismatch, thus it is not safe to create distributed cache.
Therefore for this specific occasion, we directly read from pg_dist_partition
table. However; reading from catalog is costly and we should not use this
method in other places as much as possible.
This commit fixes the problem where we incorrectly try to reach distributed table
cache when the extension is not loaded completely. We tried to reach the cache
because we wanted to get reference table information to activate the node. However
it is actually not necessary to explicitly activate the nodes which come from
master_initialize_node_metadata. Because it only runs during extension creation and
at that time there are no reference tables and all nodes are considered as active.
When we propogate the schema creation command to data nodes we add schema's
owner name too. Before this patch, we did not quote the owner's name which
causes problems with the names containing characters like '-'.
We incorrectly try to use relation cache to find particular schema's owner and
when we cannot find the schema in the relation cache(i.e always), we automatically
used current user as the schema's owner. This means we always created schemas in
the data nodes with current user. With this patch we started to use namespace
cache to find schemas.
With this commit, we start to use custom compiled PostgreSQL builds in
Travis for merge commits. This allows us to run isolation tests and
PostgreSQL's own regression tests along with our regression tests in
Travis.
Since manually compiling PostgreSQL takes more time and we also add new
tests, we only enable running these tests on merge commits.
* Accept invalidation messages before accessing the metadata cache
This commit is crucial to prevent stale metadata reads from the
cache. Without this commit, some of the operations may use stale
metadata which could end up with various bugs such as crashes,
inconsistent/lost data etc.
As an example, consider that a COPY operation is blocked on shard
metadata lock. Another concurrent session updates the metadata and
invalidates the cache. However, since Citus doesn't accept invalidations,
COPY continues with the stale metadata once it acquires the lock.
With this commit, we make sure that invalidation messages are accepted
just before accessing the metadata cache and preventing any operation to
use stale metadata.
* Add isolation tests for placement changes and conccurrent operations
- add node with reference table vs COPY/insert/update/DDL
- repair shard vs COPY/insert/update/DDL
- repair shard vs repair shard
Jason Petersen
Marco Slot
jmunsch
Onder Kalaci
Burak Yucesoy